Overclock.net › Forums › Specialty Builds › Servers › 20TB server. need help lol
New Posts  All Forums:Forum Nav:

20TB server. need help lol

post #1 of 37
Thread Starter 
hey guys. I need some help with building a server. Ive never built one before and compared to what most people need in a server mine is going to be ridiculous to say the least. I am going to need about 20TB(20,000GB) of storage(i filled up my 4TB in my gaming rig in about a week lol). Im planning to use 10x2TB HDDs. I already have an old Coolermaster HAF 932 Chassis and a MSI 790FX-GD70 Motherboard Id like to use if possible. the motherboard is an AM3 socket BTW. I can order adapters so i can put HDDs into the 5.25 inch bays which will give me a total of 10 for HDDS and 1 for a Blu-ray/DVD drive. Now thats where im going to start. can anyone recommend what other parts i need such as how much RAM or how fast the CPU should be? Just anything that you think i should do would help.
Impulse
(13 items)
 
  
CPUMotherboardGraphicsRAM
Phenom II X6 1090T Thuban Black Edition Asus Crosshair IV Formula Asus Nvidia ENGTX480 16GB Corsair Dominator (CMP8GX3M2A1600C9) 
Hard DriveOptical DriveOSMonitor
Corsair P128 SSD, SAMSUNG Spinpoint 4TB total LITE-ON Blu-ray Drive Windows 7 Ultimate 64-bit Samsung SyncMaster XL2370 (23in LED) 
KeyboardPowerCaseMouse
Razer BlackWidow Ultimate (Dragon Age II Edition) Corsair AX1200 Cooler Master HAF-X Razer Naga Epic 
Mouse Pad
Razer Ironclad 
  hide details  
Reply
Impulse
(13 items)
 
  
CPUMotherboardGraphicsRAM
Phenom II X6 1090T Thuban Black Edition Asus Crosshair IV Formula Asus Nvidia ENGTX480 16GB Corsair Dominator (CMP8GX3M2A1600C9) 
Hard DriveOptical DriveOSMonitor
Corsair P128 SSD, SAMSUNG Spinpoint 4TB total LITE-ON Blu-ray Drive Windows 7 Ultimate 64-bit Samsung SyncMaster XL2370 (23in LED) 
KeyboardPowerCaseMouse
Razer BlackWidow Ultimate (Dragon Age II Edition) Corsair AX1200 Cooler Master HAF-X Razer Naga Epic 
Mouse Pad
Razer Ironclad 
  hide details  
Reply
post #2 of 37
If it is just for storage, I don't think anything over 4GB of ram would be beneficial. As for the processor, I'm a noob at AMDs but a quad core shouldn't be too expensive, considering AMD's strong point is how cheap it is compared to intel.

EDIT: what exactly are u gonna use teh server for? Storage between a couple of computers, I'm guessing?

My $0.02.
post #3 of 37
Thread Starter 
It is going to be mainly where i store my movies and music so i can access them from any computer and because i cant fit 20TB into my gaming rig lol. pricing is not an issue however i dont want to waste my money buying a 6-core if i only need a 2-core or 4-core. same for memory.
also i dont know how im going to attach 10 HDDs to a motherboard that only has 6 SATA ports lol
Impulse
(13 items)
 
  
CPUMotherboardGraphicsRAM
Phenom II X6 1090T Thuban Black Edition Asus Crosshair IV Formula Asus Nvidia ENGTX480 16GB Corsair Dominator (CMP8GX3M2A1600C9) 
Hard DriveOptical DriveOSMonitor
Corsair P128 SSD, SAMSUNG Spinpoint 4TB total LITE-ON Blu-ray Drive Windows 7 Ultimate 64-bit Samsung SyncMaster XL2370 (23in LED) 
KeyboardPowerCaseMouse
Razer BlackWidow Ultimate (Dragon Age II Edition) Corsair AX1200 Cooler Master HAF-X Razer Naga Epic 
Mouse Pad
Razer Ironclad 
  hide details  
Reply
Impulse
(13 items)
 
  
CPUMotherboardGraphicsRAM
Phenom II X6 1090T Thuban Black Edition Asus Crosshair IV Formula Asus Nvidia ENGTX480 16GB Corsair Dominator (CMP8GX3M2A1600C9) 
Hard DriveOptical DriveOSMonitor
Corsair P128 SSD, SAMSUNG Spinpoint 4TB total LITE-ON Blu-ray Drive Windows 7 Ultimate 64-bit Samsung SyncMaster XL2370 (23in LED) 
KeyboardPowerCaseMouse
Razer BlackWidow Ultimate (Dragon Age II Edition) Corsair AX1200 Cooler Master HAF-X Razer Naga Epic 
Mouse Pad
Razer Ironclad 
  hide details  
Reply
post #4 of 37
your going to need a raid card. have you thought about what your going to use. it would be easy to setup a nas woudnt take much just to serve the files
Home
(17 items)
 
Server
(15 items)
 
 
CPUCPUMotherboardGraphics
intel Xeon 2670 intel xeon 2670 Intel S2600CP Evga 980ti 
RAMHard DriveHard DriveOptical Drive
64gb Ecc 24x 128gb ssds  Crucial Mx1000 Asus 
OSMonitorMonitorKeyboard
Windows 10 Pro Sony XBR43X830C 43" 4K Monoprice 28" 4k Max Keyboards NightHawk X8 Custom 
PowerCaseMouseMouse Pad
Evga Supernova 850w  Norco Rpc-4224 Logitech G900 Chaos Steelseries QCK 
Audio
FIIO E7/E9 combo 
CPUCPUMotherboardRAM
Intel L5640 Intel L5640  Supermicro - X8DT3 24Gb ECC 
Hard DriveHard DriveHard DriveOS
4x 128gb ssd's raid 0 - Cache Wd Red 3tb Seagate 8tb smr - parity Unriad 6.2 
PowerCaseOtherOther
ThermalTake 700 watt  Norco 4220 HP sas expander Lsi 9211-8i 
  hide details  
Reply
Home
(17 items)
 
Server
(15 items)
 
 
CPUCPUMotherboardGraphics
intel Xeon 2670 intel xeon 2670 Intel S2600CP Evga 980ti 
RAMHard DriveHard DriveOptical Drive
64gb Ecc 24x 128gb ssds  Crucial Mx1000 Asus 
OSMonitorMonitorKeyboard
Windows 10 Pro Sony XBR43X830C 43" 4K Monoprice 28" 4k Max Keyboards NightHawk X8 Custom 
PowerCaseMouseMouse Pad
Evga Supernova 850w  Norco Rpc-4224 Logitech G900 Chaos Steelseries QCK 
Audio
FIIO E7/E9 combo 
CPUCPUMotherboardRAM
Intel L5640 Intel L5640  Supermicro - X8DT3 24Gb ECC 
Hard DriveHard DriveHard DriveOS
4x 128gb ssd's raid 0 - Cache Wd Red 3tb Seagate 8tb smr - parity Unriad 6.2 
PowerCaseOtherOther
ThermalTake 700 watt  Norco 4220 HP sas expander Lsi 9211-8i 
  hide details  
Reply
post #5 of 37
Thread Starter 
no idea what "nas" even means lol. im a total noob when it comes to servers =P
Impulse
(13 items)
 
  
CPUMotherboardGraphicsRAM
Phenom II X6 1090T Thuban Black Edition Asus Crosshair IV Formula Asus Nvidia ENGTX480 16GB Corsair Dominator (CMP8GX3M2A1600C9) 
Hard DriveOptical DriveOSMonitor
Corsair P128 SSD, SAMSUNG Spinpoint 4TB total LITE-ON Blu-ray Drive Windows 7 Ultimate 64-bit Samsung SyncMaster XL2370 (23in LED) 
KeyboardPowerCaseMouse
Razer BlackWidow Ultimate (Dragon Age II Edition) Corsair AX1200 Cooler Master HAF-X Razer Naga Epic 
Mouse Pad
Razer Ironclad 
  hide details  
Reply
Impulse
(13 items)
 
  
CPUMotherboardGraphicsRAM
Phenom II X6 1090T Thuban Black Edition Asus Crosshair IV Formula Asus Nvidia ENGTX480 16GB Corsair Dominator (CMP8GX3M2A1600C9) 
Hard DriveOptical DriveOSMonitor
Corsair P128 SSD, SAMSUNG Spinpoint 4TB total LITE-ON Blu-ray Drive Windows 7 Ultimate 64-bit Samsung SyncMaster XL2370 (23in LED) 
KeyboardPowerCaseMouse
Razer BlackWidow Ultimate (Dragon Age II Edition) Corsair AX1200 Cooler Master HAF-X Razer Naga Epic 
Mouse Pad
Razer Ironclad 
  hide details  
Reply
post #6 of 37
NAS = Networked Attached Storage, but that's usually a stand alone box. And I have never seen them in a 20TB size.... though there might be one out there

Edit: Something like this would add the extra 4 ports you're missing. Note that you would have no redundancy (RAID) A RAID 1 would give you half the storage (20TB of space leaves you with 10TB of redundant usable space)

http://ncix.com/products/?sku=54138&...20Technologies

or two of these:
http://www.newegg.ca/Product/Product...-008-_-Product

And a quad core CPU with 4 GB RAM should be plenty if it's just for a file server. You might want to double check that you have a GB network card on that mother board.
Edited by fishman78 - 1/29/11 at 7:21am
    
CPUMotherboardGraphicsRAM
Intel Core i7 3820 @ 4.3GHz Gigabyte x79-UD3 MSI GTX 570 4x4GB Corsair Vengeance - Quad Channel 
Hard DriveHard DriveOptical DriveCooling
256GB Vertex 4 Seagate 1 TB 7200 RMP LG Multi Corsair H100 
OSMonitorKeyboardPower
Windows 7 Ultimate 64bit BenQ GW2750 Razer BlackWidow Tournament 10 key less Corsair 850AX 
CaseMouseMouse PadAudio
Obsidian 800D G500 XFX WarPad OMB 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Intel Core i7 3820 @ 4.3GHz Gigabyte x79-UD3 MSI GTX 570 4x4GB Corsair Vengeance - Quad Channel 
Hard DriveHard DriveOptical DriveCooling
256GB Vertex 4 Seagate 1 TB 7200 RMP LG Multi Corsair H100 
OSMonitorKeyboardPower
Windows 7 Ultimate 64bit BenQ GW2750 Razer BlackWidow Tournament 10 key less Corsair 850AX 
CaseMouseMouse PadAudio
Obsidian 800D G500 XFX WarPad OMB 
  hide details  
Reply
post #7 of 37
Yeah, if you're looking for an OS, probably use FreeNAS.
As a CPU, I would say, I dunno, Athlon X4 620? and 4 gigs of ram should be fine. You can probably buy the cheapest ram you can find, you won't need great timings.
Also, you probably will need a new SATA PCI card to expand your SATA openings (I think you have 6 ports on the mobo)
The Behemoth
(13 items)
 
  
CPUMotherboardGraphicsRAM
1090t @ 4.2 MSI 890FXA-GD70 Gigabyte non-reference 5850 4gb OCZ Platinum DDR3 
Hard DriveOptical DriveOSMonitor
2x 500gb Spinpoint F3 MSI DVD-RW Ubuntu 10.10 I-INC 25", Asus 24" 
KeyboardPowerCaseMouse
Das Model S Professional Ultra LSP 750w 700D Purple Logitech 
  hide details  
Reply
The Behemoth
(13 items)
 
  
CPUMotherboardGraphicsRAM
1090t @ 4.2 MSI 890FXA-GD70 Gigabyte non-reference 5850 4gb OCZ Platinum DDR3 
Hard DriveOptical DriveOSMonitor
2x 500gb Spinpoint F3 MSI DVD-RW Ubuntu 10.10 I-INC 25", Asus 24" 
KeyboardPowerCaseMouse
Das Model S Professional Ultra LSP 750w 700D Purple Logitech 
  hide details  
Reply
post #8 of 37
Quote:
Originally Posted by xXxT0MBstonexXx View Post
It is going to be mainly where i store my movies and music so i can access them from any computer and because i cant fit 20TB into my gaming rig lol. pricing is not an issue however i dont want to waste my money buying a 6-core if i only need a 2-core or 4-core. same for memory.
also i dont know how im going to attach 10 HDDs to a motherboard that only has 6 SATA ports lol
Heya,

I have this setup, essentially. I have a basic AM3 AMD board with a dualcore and 2Gigs of DDR3 ram as the basis of my movie server. It is powerful enough to playback movies on its own, but I tend to simply stream from it (both video and mounting ISO from it over my network, both wired and wireless). Works fine both ways. I run Windows Server 2008 R2 on it because it's free from Dreamspark if you have an *.edu email address. I simply installed packages and made it more of a workstation than a server operating system, so that it is much more like using your typical Windows 7 operating system instead of a `server'. The word server in things tends to make it scary and big, but in reality, the word is over used. Anything that serves things out is a server.

I have 14 TB in my server right now. And I use 1:1 redundancy. That means I have the same data set on two separate hard drives. With 2TB drives as cheap as $70 (those samsung deals have been so good lately), I'd rather have $140 for 2TB worth of redundant space. I for one am not going to sit and re-backup all my movies again (which is in the hundreds, likely what you're gearing up to do, involving DVD's and BluRay). I do my redundancy manually. Meaning, I just copy the same data to two drives. One drive is for accessing and is shared on my network. The other drive, it's "mirror", is not shared and is not accessed at all unless I'm adding data to it. What this accomplishes is that the mirror drive doesn't even spin up because it's not accessed at all, while the drive it mirrors is accessed all the time and remains spinning. Less ware on my mirror drive. You can't do that with RAID, which is why I don't use RAID1 for example. And this isn't special software. This is just two drives, I copy to both, share one, don't share the other. Simple as that. Not everyone needs redundancy. But when your data set is in the multiple terabyte range, I for one, do not want to re-copy a horde of data from DVD's & BluRays which takes forever compared to simply replacing a drive and copying it at HDD sata speed (60+ megabytes per second) from one drive to another in one click (no swapping of 100's of discs, copy time, encode, etc). So if you're in this camp you're going to want redundnacy. I recommend simple 1:1 like I described here. Do NOT even consider RAID5 or RAID6 here. Your data set is too high for them and the cost is not worth it (it's still a failed array easily at 20TB). I suggest simple groups of 1:1 redundancy.

Alternative method: Windows Home Server's method. It basically does what I just described, but it shares based on folders. The data is then simply assigned to another physical drive so it has redundancy. That way the whole drive isn't `redundant' but only folders you wish to have backups of. This helps save space. But again, you can do this manually yourself for free. I do it myself. I don't like automation when it comes to copying and backing up massive data sets that when lost or screwed up result in hours and days of fixing time.

The motherboard cannot of course accept every single HDD sata connection, it has limits. This is why you have PCIe and PCI slots. You add SATA controller cards. And here's the lovely thing about it: you do not need RAID, you do not need SATA2.0 or even SATA3.0. Basic SATA will work perfectly fine. A single 5400~7200 rpm drive is not going to even touch saturation of basic SATA, let alone SATA2 and SATA3. So don't think you need expensive stuff. A cheap $30 controller that adds 4 SATA ports on PCI is more than enough. And PCI bandwidth is still higher than a basic hard drive can saturate. You will not be limited. I use the 5400rpm mass storage drives on basic SATA over my gigabit network and I still get about 68 megabytes per second transfer speeds when copying. Your drives cannot even saturate gigabit network (wired). On wireless, on the other hand, the wireless transfer speed is slower than what a drive can do--so you always have a bottleneck there for consumer level stuff. But again I can't stress it enough: you do not need high end stuff. You will not saturate basic SATA with these drives.

All you need:

Operating system. I suggest Windows Home Server (not free) or Windows Server 2008 R2 (free if you go to Dreamspark and have *.edu email address), otherwise, you can use Linux for free. For pay systems, simply use Windows 7 (any version) and it will do what you need just fine or Windows Home Server.

PCI (or PCIe) SATA controller cards. No RAID. Nothing fancy. Not SATA2 or SATA3 even. Just basic SATA. It's cheap and you will not saturate the bandwidth (your drives do not copy sustained rates over 150 megabytes per second--note that number is a dirty estimate, but it's still nearly twice what your sustained copy speed will be on a 5400~7200rpm HDD will be; do NOT base your bandwidth needs on burst speeds, base it on sustained read/write speeds). You can get cheap ones that have 4 ports for $30 or so. That means you can easily add tons of drives to whatever motherboard you have. Spend more money and get more ports on a single card if you please.

Gigabit router/switch, gigabit capable network cables. Otherwise, a wireless N or better router and appropriate wifi cards for devices.

You can operate your server wirelessly, headless, depending on what operating system you use. Mine is headless. I access it and control it using TightVNC (google that if you're interested).

Good luck!

Very best,
post #9 of 37
Thread Starter 
wow MalVeauX. thanks for the time you spent writing that lol. good to see someone has already done what im aiming for. can you help me with a parts list please? I know thats asking alot. I build custom PCs as a side job so i know just how much trouble goes into making a parts list but im in the dark on this one. Ive come up with some parts so tell me if you think its overkill. I prefer to get "value" instead of the cheapest price and this is what i came up with.

Im going to use these as my HDDs: http://www.newegg.com/Product/Produc...82E16822152245
Im only going to buy 4 of them to start and add more as i need them.

http://www.newegg.com/Product/Produc...82E16819103817 for my CPU

http://www.newegg.com/Product/Produc...82E16820145251 for my RAM

as for the SATA/RAID card i have no idea what companies are good

now the only thing im worried about is if my motherboard even works lol. its been sitting in my HAF 932 case for about 2 years now with the side panels off and buried under dust lol
Edited by xXxT0MBstonexXx - 1/29/11 at 7:52am
Impulse
(13 items)
 
  
CPUMotherboardGraphicsRAM
Phenom II X6 1090T Thuban Black Edition Asus Crosshair IV Formula Asus Nvidia ENGTX480 16GB Corsair Dominator (CMP8GX3M2A1600C9) 
Hard DriveOptical DriveOSMonitor
Corsair P128 SSD, SAMSUNG Spinpoint 4TB total LITE-ON Blu-ray Drive Windows 7 Ultimate 64-bit Samsung SyncMaster XL2370 (23in LED) 
KeyboardPowerCaseMouse
Razer BlackWidow Ultimate (Dragon Age II Edition) Corsair AX1200 Cooler Master HAF-X Razer Naga Epic 
Mouse Pad
Razer Ironclad 
  hide details  
Reply
Impulse
(13 items)
 
  
CPUMotherboardGraphicsRAM
Phenom II X6 1090T Thuban Black Edition Asus Crosshair IV Formula Asus Nvidia ENGTX480 16GB Corsair Dominator (CMP8GX3M2A1600C9) 
Hard DriveOptical DriveOSMonitor
Corsair P128 SSD, SAMSUNG Spinpoint 4TB total LITE-ON Blu-ray Drive Windows 7 Ultimate 64-bit Samsung SyncMaster XL2370 (23in LED) 
KeyboardPowerCaseMouse
Razer BlackWidow Ultimate (Dragon Age II Edition) Corsair AX1200 Cooler Master HAF-X Razer Naga Epic 
Mouse Pad
Razer Ironclad 
  hide details  
Reply
post #10 of 37
Quote:
Originally Posted by MalVeauX View Post
Heya,

I have this setup, essentially. I have a basic AM3 AMD board with a dualcore and 2Gigs of DDR3 ram as the basis of my movie server. It is powerful enough to playback movies on its own, but I tend to simply stream from it (both video and mounting ISO from it over my network, both wired and wireless). Works fine both ways. I run Windows Server 2008 R2 on it because it's free from Dreamspark if you have an *.edu email address. I simply installed packages and made it more of a workstation than a server operating system, so that it is much more like using your typical Windows 7 operating system instead of a `server'. The word server in things tends to make it scary and big, but in reality, the word is over used. Anything that serves things out is a server.

I have 14 TB in my server right now. And I use 1:1 redundancy. That means I have the same data set on two separate hard drives. With 2TB drives as cheap as $70 (those samsung deals have been so good lately), I'd rather have $140 for 2TB worth of redundant space. I for one am not going to sit and re-backup all my movies again (which is in the hundreds, likely what you're gearing up to do, involving DVD's and BluRay). I do my redundancy manually. Meaning, I just copy the same data to two drives. One drive is for accessing and is shared on my network. The other drive, it's "mirror", is not shared and is not accessed at all unless I'm adding data to it. What this accomplishes is that the mirror drive doesn't even spin up because it's not accessed at all, while the drive it mirrors is accessed all the time and remains spinning. Less ware on my mirror drive. You can't do that with RAID, which is why I don't use RAID1 for example. And this isn't special software. This is just two drives, I copy to both, share one, don't share the other. Simple as that. Not everyone needs redundancy. But when your data set is in the multiple terabyte range, I for one, do not want to re-copy a horde of data from DVD's & BluRays which takes forever compared to simply replacing a drive and copying it at HDD sata speed (60+ megabytes per second) from one drive to another in one click (no swapping of 100's of discs, copy time, encode, etc). So if you're in this camp you're going to want redundnacy. I recommend simple 1:1 like I described here. Do NOT even consider RAID5 or RAID6 here. Your data set is too high for them and the cost is not worth it (it's still a failed array easily at 20TB). I suggest simple groups of 1:1 redundancy.

Alternative method: Windows Home Server's method. It basically does what I just described, but it shares based on folders. The data is then simply assigned to another physical drive so it has redundancy. That way the whole drive isn't `redundant' but only folders you wish to have backups of. This helps save space. But again, you can do this manually yourself for free. I do it myself. I don't like automation when it comes to copying and backing up massive data sets that when lost or screwed up result in hours and days of fixing time.

The motherboard cannot of course accept every single HDD sata connection, it has limits. This is why you have PCIe and PCI slots. You add SATA controller cards. And here's the lovely thing about it: you do not need RAID, you do not need SATA2.0 or even SATA3.0. Basic SATA will work perfectly fine. A single 5400~7200 rpm drive is not going to even touch saturation of basic SATA, let alone SATA2 and SATA3. So don't think you need expensive stuff. A cheap $30 controller that adds 4 SATA ports on PCI is more than enough. And PCI bandwidth is still higher than a basic hard drive can saturate. You will not be limited. I use the 5400rpm mass storage drives on basic SATA over my gigabit network and I still get about 68 megabytes per second transfer speeds when copying. Your drives cannot even saturate gigabit network (wired). On wireless, on the other hand, the wireless transfer speed is slower than what a drive can do--so you always have a bottleneck there for consumer level stuff. But again I can't stress it enough: you do not need high end stuff. You will not saturate basic SATA with these drives.

All you need:

Operating system. I suggest Windows Home Server (not free) or Windows Server 2008 R2 (free if you go to Dreamspark and have *.edu email address), otherwise, you can use Linux for free. For pay systems, simply use Windows 7 (any version) and it will do what you need just fine or Windows Home Server.

PCI (or PCIe) SATA controller cards. No RAID. Nothing fancy. Not SATA2 or SATA3 even. Just basic SATA. It's cheap and you will not saturate the bandwidth (your drives do not copy sustained rates over 150 megabytes per second--note that number is a dirty estimate, but it's still nearly twice what your sustained copy speed will be on a 5400~7200rpm HDD will be; do NOT base your bandwidth needs on burst speeds, base it on sustained read/write speeds). You can get cheap ones that have 4 ports for $30 or so. That means you can easily add tons of drives to whatever motherboard you have. Spend more money and get more ports on a single card if you please.

Gigabit router/switch, gigabit capable network cables. Otherwise, a wireless N or better router and appropriate wifi cards for devices.

You can operate your server wirelessly, headless, depending on what operating system you use. Mine is headless. I access it and control it using TightVNC (google that if you're interested).

Good luck!

Very best,
Man what a post. Not my thread but that deserves a rep. Anyways your server will be fine with 4GB ram max and a dual core cpu.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Servers
Overclock.net › Forums › Specialty Builds › Servers › 20TB server. need help lol