Overclock.net › Forums › Components › Hard Drives & Storage › RAID Controllers and Software › 72TB FreeNAS Build Advice
New Posts  All Forums:Forum Nav:

72TB FreeNAS Build Advice

post #1 of 4
10/28/12 at 9:50pm
Thread Starter 
I've been doing some research for a while now for a FreeNAS build and haven't been able to come up with anything else on my own. I'm hoping to get some feedback on this. Please feel free to let me know what you would change and why.

This system is going to be used to store video files. We will not have editors edit from this NAS. We might have 4 users that just drop/grab files from it at random times.

Here are a couple of questions.

1. FreeNAS seems to like memory and this motherboard only supports 32GB. They recommend about 1GB per 1TB. I've looked at dual CPU motherboards for the E5260 that allow more memory but the processors are older and people tell me to go with newer faster processors. What's your take on this? I'm willing to spend $1000 on a good combo of MB/CPU. Would this affect any rebuild time or performance? Should I go six core or 2x quad core?

2. I'm not sure how comfortable I am with the thumb drive for booting. Should I Raid1 FreeNAS with 2 SSD's? I know FreeNAS/ZFS can operate on anything but I just want a solid system.

Purchase Plan:

Supermicro - 36 Server Chassis - $1650
http://www.newegg.com/Product/Product.aspx?Item=N82E16811152540

Processor - $340
http://www.newegg.com/Product/Product.aspx?Item=N82E16819115081

Motherboard - $199
http://www.newegg.com/Product/Product.aspx?Item=N82E16813182253

Memory 32GB Total - $ 268
http://www.superbiiz.com/detail.php?name=D38GE1600S

Raid Card x 3 - $775
http://www.newegg.com/Product/Product.aspx?Item=N82E16816118112

SSD - OS Mirror
http://www.newegg.com/Product/Product.aspx?Item=N82E16820148442

Hard Drives x 24 - $5,040
http://www.newegg.com/Product/Product.aspx?item=N82E16822236344

NIC - $135
http://www.newegg.com/Product/Product.aspx?Item=N82E16833106015


Thanks
Reply
Reply
post #2 of 4
11/5/12 at 2:48am
If it's a production server i would definitely raid 1 the OS drives. But I think 128GB is a waste. I'd get the smallest SSD you can for the OS mirror.

The rest looks great. FreeNAS will eat every GB of RAM you give it for cache. One way you could get round this 32GB limit is to get a fast SSD and use it for a cache/log device (depending on whether you value read or write speeds more).

It's called "L2ARC" and there's a great read of it here and here.

I suppose you could have 4 SSDs. 2 as a mirror, one cache, and one log. If one of your L2ARC drives goes down, the server will keep operating but at slower speeds. However it's worth noting that if you don't have a UPS and the power fails, you'll loose the data that's been written to the log drive but hasn't been committed to the pool yet. So i'd definitely have a UPS on it.
The little difference engine that could
(16 items)
 
Big Bertha: The 4u File Server
(13 items)
  		 
CPUMotherboardGraphicsRAM
2500K (L041C124) | 5.3GHz 1.47v Asus P8P67 Asus GTX480 | 880/1760/2000 8GB CorsairXMS3 1600MHz 
Hard DriveHard DriveCoolingCooling
80GB Intel x25-M 1TB Spinning (wheeee) RS 360, AP15s,  EK Full Cover GPU Block 
CoolingOSMonitorKeyboard
XSPC CPU Block Windows 7 64 Bit U2713 + U2311 Filco Majestouch blank tenkeyless 
PowerCaseMouse
Corsair TX750 Corsair 700D Logitech G500 
View all
CPUMotherboardRAMHard Drive
Intel i3 530 Asus P7P55-m 4GB DDR3 Seagate 7200RPM 2TB 
Hard DriveHard DriveHard DriveHard Drive
Samsung SP 1TB 5400RPM Seagate 7200RPM 2TB Seagate 7200RPM 2TB Seagate 7200RPM 2TB 
OSPowerCase
unRAID Corsair CX430 xcase RM400 4U 
View all
  hide details  
Reply
The little difference engine that could
(16 items)
 
Big Bertha: The 4u File Server
(13 items)
  		 
CPUMotherboardGraphicsRAM
2500K (L041C124) | 5.3GHz 1.47v Asus P8P67 Asus GTX480 | 880/1760/2000 8GB CorsairXMS3 1600MHz 
Hard DriveHard DriveCoolingCooling
80GB Intel x25-M 1TB Spinning (wheeee) RS 360, AP15s,  EK Full Cover GPU Block 
CoolingOSMonitorKeyboard
XSPC CPU Block Windows 7 64 Bit U2713 + U2311 Filco Majestouch blank tenkeyless 
PowerCaseMouse
Corsair TX750 Corsair 700D Logitech G500 
View all
CPUMotherboardRAMHard Drive
Intel i3 530 Asus P7P55-m 4GB DDR3 Seagate 7200RPM 2TB 
Hard DriveHard DriveHard DriveHard Drive
Samsung SP 1TB 5400RPM Seagate 7200RPM 2TB Seagate 7200RPM 2TB Seagate 7200RPM 2TB 
OSPowerCase
unRAID Corsair CX430 xcase RM400 4U 
View all
  hide details  
Reply
post #3 of 4
11/5/12 at 8:25pm
Thread Starter 
Thanks for the link Boyboyd, I'm definitely going to look over all these options. The more I read about ZFS, the more I'm blown away by how cool it is. I do have a question on a network card and a switch. What do you think of these 2.

I was thinking of getting this card and this switch for LACP.

http://www.newegg.com/Product/Product.aspx?Item=N82E16833106050

http://www.newegg.com/Product/Product.aspx?Item=N82E16833122074

10Gigabit is just too expensive for now
Reply
Reply
post #4 of 4
11/6/12 at 3:49am
Quote:
Originally Posted by FreeMe View Post

Thanks for the link Boyboyd, I'm definitely going to look over all these options. The more I read about ZFS, the more I'm blown away by how cool it is. I do have a question on a network card and a switch. What do you think of these 2.
I was thinking of getting this card and this switch for LACP.
http://www.newegg.com/Product/Product.aspx?Item=N82E16833106050
http://www.newegg.com/Product/Product.aspx?Item=N82E16833122074
10Gigabit is just too expensive for now

I've never actually used managed switches before, but i know that card is good. Everyone seems to use intel cards because they're compatible with everything.

ZFS is incredible. The reason i don't use it on my home nas is that it's very difficult to expand an array, but if you're starting out with all the drives you'll have (like you are) then i'd use it in a heatbeat.
The little difference engine that could
(16 items)
 
Big Bertha: The 4u File Server
(13 items)
  		 
CPUMotherboardGraphicsRAM
2500K (L041C124) | 5.3GHz 1.47v Asus P8P67 Asus GTX480 | 880/1760/2000 8GB CorsairXMS3 1600MHz 
Hard DriveHard DriveCoolingCooling
80GB Intel x25-M 1TB Spinning (wheeee) RS 360, AP15s,  EK Full Cover GPU Block 
CoolingOSMonitorKeyboard
XSPC CPU Block Windows 7 64 Bit U2713 + U2311 Filco Majestouch blank tenkeyless 
PowerCaseMouse
Corsair TX750 Corsair 700D Logitech G500 
View all
CPUMotherboardRAMHard Drive
Intel i3 530 Asus P7P55-m 4GB DDR3 Seagate 7200RPM 2TB 
Hard DriveHard DriveHard DriveHard Drive
Samsung SP 1TB 5400RPM Seagate 7200RPM 2TB Seagate 7200RPM 2TB Seagate 7200RPM 2TB 
OSPowerCase
unRAID Corsair CX430 xcase RM400 4U 
View all
  hide details  
Reply
The little difference engine that could
(16 items)
 
Big Bertha: The 4u File Server
(13 items)
  		 
CPUMotherboardGraphicsRAM
2500K (L041C124) | 5.3GHz 1.47v Asus P8P67 Asus GTX480 | 880/1760/2000 8GB CorsairXMS3 1600MHz 
Hard DriveHard DriveCoolingCooling
80GB Intel x25-M 1TB Spinning (wheeee) RS 360, AP15s,  EK Full Cover GPU Block 
CoolingOSMonitorKeyboard
XSPC CPU Block Windows 7 64 Bit U2713 + U2311 Filco Majestouch blank tenkeyless 
PowerCaseMouse
Corsair TX750 Corsair 700D Logitech G500 
View all
CPUMotherboardRAMHard Drive
Intel i3 530 Asus P7P55-m 4GB DDR3 Seagate 7200RPM 2TB 
Hard DriveHard DriveHard DriveHard Drive
Samsung SP 1TB 5400RPM Seagate 7200RPM 2TB Seagate 7200RPM 2TB Seagate 7200RPM 2TB 
OSPowerCase
unRAID Corsair CX430 xcase RM400 4U 
View all
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: RAID Controllers and Software
Overclock.net › Forums › Components › Hard Drives & Storage › RAID Controllers and Software › 72TB FreeNAS Build Advice