Overclock.net › Forums › Specialty Builds › Servers › Frustrated with FreeNAS/ZFS performance...What now?
New Posts  All Forums:Forum Nav:

Frustrated with FreeNAS/ZFS performance...What now?

post #1 of 36
Thread Starter 
So I recently built a FreeNAS server (first time using ZFS) and as the title of the thread reads I'm less than satisfied to say the least. My previous file server was built on an 8-drive RAID10 array and the performance was excellent (fully saturating my gigabit connection). Now I'm getting 50-60MB/s tops (many times much lower) write performance over CIFS with my RAIDz2 array. I knew I'd take a write hit with parity RAID but I guess I didn't realize it would be this bad.

So, with all things to consider my mind has been wondering the past week. Stick with FreeNAS (which I admit is a lot more complex to configure than I gave it credit for) or use the hardware I built my FreeNAS server with to go a different direction?

My server specs are as follows:

CPU: Intel i3-4130
MoBo: SuperMicro X10SLM-F-O
RAM: 16GB Kingston ECC RAM
HBA: IBM ServerRAID M1015
Drives: 6x 3TB WD Reds

Additional Storage:

16GB San Disk Boot USB
128GB Crucial M4 SATA3 SSD (not currently being used)

I'm a Windows guy first and foremost. I realize Windows has it's limitations and drawbacks and inefficiencies compared to some other OS's that are for more targeted services but I can't ignore the fact that it's what I know best and the more familiar with the OS I am the better off I'm going to be without a doubt. However this server is STRICTLY for storage. I will be doing NOTHING but streaming media (via shares from this box, not services as those will be on another box) and doing backups via iSCSI. So the idea of putting something like WS 2012 on this box just seems very ineffecient to me (or convince me otherwise??). I'd also strongly prefer to keep the 16GB Flash drive as my boot drive.

Really just looking for some suggestions to be thrown out there. In the meantime I'll keep plugging along trying to make myself more familiar with FreeNAS.
post #2 of 36
ZFS is notoriously hard on memory, IOPS and throughput are well known to be directly correlated to the amount of memory that you have. Using the SSD you already have for logs and cache should better performance, although from what I understand it'll probably be more IOPS than throughput.
Indigo Hammer
(5 items)
 
Blue Danube
(7 items)
 
 
CPUGraphicsOSKeyboard
Core i5-4250U Intel HD5000 OS X 10.8.4 Filco Minila Air - MX Blue 
Audio
TCA Gizmo > Axiom Audio M2ti 
  hide details  
Reply
Indigo Hammer
(5 items)
 
Blue Danube
(7 items)
 
 
CPUGraphicsOSKeyboard
Core i5-4250U Intel HD5000 OS X 10.8.4 Filco Minila Air - MX Blue 
Audio
TCA Gizmo > Axiom Audio M2ti 
  hide details  
Reply
post #3 of 36
Thread Starter 
Quote:
Originally Posted by max302 View Post

ZFS is notoriously hard on memory, IOPS and throughput are well known to be directly correlated to the amount of memory that you have. Using the SSD you already have for logs and cache should better performance, although from what I understand it'll probably be more IOPS than throughput.

Yea I know about ZFS being a memory hog which is why I got the 16GB which should be plenty for what I'm doing. I will setup the SSD as a cache drive though and see if it helps at all.
post #4 of 36
I can at least say that with 32GB of RAM, I could fully saturate my gigabit network over CIFS and 6 WD red 3TB in RAIDZ2. I'm using vanilla FreeBSD though and use an ssd for the OS(not as a cache drive).
I now have double the drives and still no problem saturating the connection so I would think 16 should be enough and that the problem lies elsewhere, but I'm no expert so I have no idea where.
Other than the OS drive, the only other big difference would be that my CPU is much faster since I use it for a lot more stuff than a simple NAS so I would make sure the CPU usage isn't hitting 100%(I doubt it would though since it should be plenty fast for a NAS).
First decent rig
(19 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 2600K P67 Extreme4 NVIDIA GeForce GTX 560 Ti Corsair Vengeance Black 
RAMRAMRAMHard Drive
Corsair Vengeance Black Corsair Vengeance Red Corsair Vengeance Red OCZ Vertex 3 60GB 
Hard DriveOptical DriveCoolingOS
Western Digital Caviar Black 1TB ASUS 12x BD-ROM XIGMATEK LOKI SD963 Windows 7 SP1 x64 Ultimate 
MonitorKeyboardPowerCase
Sony Bravia 32" 720p 60Hz Sidewinder X4 OCZ ZX Series 850W COOLER MASTER HAF 922 
MouseMouse PadAudio
RAZER ABYSSUS Black 3 Buttons Random anime character ASUS Xonar D1 
  hide details  
Reply
First decent rig
(19 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 2600K P67 Extreme4 NVIDIA GeForce GTX 560 Ti Corsair Vengeance Black 
RAMRAMRAMHard Drive
Corsair Vengeance Black Corsair Vengeance Red Corsair Vengeance Red OCZ Vertex 3 60GB 
Hard DriveOptical DriveCoolingOS
Western Digital Caviar Black 1TB ASUS 12x BD-ROM XIGMATEK LOKI SD963 Windows 7 SP1 x64 Ultimate 
MonitorKeyboardPowerCase
Sony Bravia 32" 720p 60Hz Sidewinder X4 OCZ ZX Series 850W COOLER MASTER HAF 922 
MouseMouse PadAudio
RAZER ABYSSUS Black 3 Buttons Random anime character ASUS Xonar D1 
  hide details  
Reply
post #5 of 36
I would look into cpu utilization also. Samba on FreeNas should use a single core per connection. So clock speed is important. 3.4 Ghz should be plenty of clock speed for that. Are you using encryption, compression or dedup?
post #6 of 36
You didn't mention what version of FreeNAS you're currently running. v9.1.1 just recently came out. If you're running one of the previous versions, try upgrading the installed distro. I'm about to fire one up as a NAS datastore for my ESXi whiteboxes. Would like to compare notes with our results as applicable.
post #7 of 36
Thread Starter 
Quote:
Originally Posted by SamKook View Post

I can at least say that with 32GB of RAM, I could fully saturate my gigabit network over CIFS and 6 WD red 3TB in RAIDZ2. I'm using vanilla FreeBSD though and use an ssd for the OS(not as a cache drive).
I now have double the drives and still no problem saturating the connection so I would think 16 should be enough and that the problem lies elsewhere, but I'm no expert so I have no idea where.
Other than the OS drive, the only other big difference would be that my CPU is much faster since I use it for a lot more stuff than a simple NAS so I would make sure the CPU usage isn't hitting 100%(I doubt it would though since it should be plenty fast for a NAS).

Quote:
Originally Posted by SweetAndLow View Post

I would look into cpu utilization also. Samba on FreeNas should use a single core per connection. So clock speed is important. 3.4 Ghz should be plenty of clock speed for that. Are you using encryption, compression or dedup?

CPU utilization is less than 50% during large file transfers. I'm not doing any encryption, compression or dedup.
Quote:
Originally Posted by ButlerKevinD View Post

You didn't mention what version of FreeNAS you're currently running. v9.1.1 just recently came out. If you're running one of the previous versions, try upgrading the installed distro. I'm about to fire one up as a NAS datastore for my ESXi whiteboxes. Would like to compare notes with our results as applicable.

I'm running the latest version. I'm not using my FreeNAS box as VM datastore though as I have an SSD in my VM box for that and the speed is just no comparison. I do want to use my FreeNAS box for VM backups though.
post #8 of 36
I would ditch using Freenas entirely if performance is ANY consideration. Go download Openindiana and install Napp-it on top of it. I run this off an AMD E-350 integrated board with 8GB and x4 500gb barracuda drives (generation 7-8) and I am able to pretty close to saturate my gigabit line, there is no reason your configuration shouldn't.

I've scaled this setup from little boxes like the one I use to 10k builds for clients, Openindiana has been rock stable with ZFS and Napp-it web console makes it a breeze to manage.

Freenas is great and easy to set up, but performance is utter junk.
post #9 of 36
Thread Starter 
Quote:
Originally Posted by swat565 View Post

I would ditch using Freenas entirely if performance is ANY consideration. Go download Openindiana and install Napp-it on top of it. I run this off an AMD E-350 integrated board with 8GB and x4 500gb barracuda drives (generation 7-8) and I am able to pretty close to saturate my gigabit line, there is no reason your configuration shouldn't.

I've scaled this setup from little boxes like the one I use to 10k builds for clients, Openindiana has been rock stable with ZFS and Napp-it web console makes it a breeze to manage.

Freenas is great and easy to set up, but performance is utter junk.

Are the hardware requirements similar/the same for OpenIndiana as they are for FreeNAS?
post #10 of 36
To clarify, did you go from a 4 * 2-disk mirror to a 1 * 8-disk raidz2?
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Servers
Overclock.net › Forums › Specialty Builds › Servers › Frustrated with FreeNAS/ZFS performance...What now?