Overclock.net › Forums › Specialty Builds › Servers › Question on massive I/O on a server
New Posts  All Forums:Forum Nav:

Question on massive I/O on a server

post #1 of 13
Thread Starter 
Hey I am looking at servers for one of our departments at work. The machine will essentially store logs at around 4 gbits a second but we are looking for a long term solution that could go up to 10 gigabits per second. We have narrowed down most of the parts we need, but in terms of HDDs we are still unsure what could be the bottleneck.

I was thinking of either doing a raid array or introducing an SSD that would get backed up every night onto a regular spin plate 7200. I am not sure which is more efficient. Any input on storing large amounts of data on the fly would be great. These are going to be standard PCAP files but they will be of an entire network so the throughput will be enormous. Cheers.
post #2 of 13
So essentially you're packet capturing your network? Just to clarify.....

There are products for that. Extrahop being the first that comes to mind, really cool stuff, but to be honest if you have 4 gbit a second coming in, don't know why you're trying a home built solution that has no support or guarantees...
post #3 of 13
I never recommend homebuilt machines of any kind, let alone servers, in a business environment. Homebuilding a machine that processes data that quickly would be suicidal.

Anyway... how's your switching? Is this going to be FC or 10gbe, or some kind of 4x1gb setup?
    
CPUMotherboardGraphicsRAM
2x intel Xeon E5-2650 Supermicro MBD-X9DR3-F-O Onboard awesomeness 8 x 8GB Kingston DDR3 1333 ECC 
Hard DriveCoolingOSMonitor
4x WD Green 2TB in RAID 10 2x Coolermaster Hyper 212 EVOs Windows Server 2012 Datacenter 3x Dell Ultrasharp U2410s 
PowerCase
Corsair AX1200 Case Labs TX10-D 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
2x intel Xeon E5-2650 Supermicro MBD-X9DR3-F-O Onboard awesomeness 8 x 8GB Kingston DDR3 1333 ECC 
Hard DriveCoolingOSMonitor
4x WD Green 2TB in RAID 10 2x Coolermaster Hyper 212 EVOs Windows Server 2012 Datacenter 3x Dell Ultrasharp U2410s 
PowerCase
Corsair AX1200 Case Labs TX10-D 
  hide details  
Reply
post #4 of 13
Thread Starter 
Just to clarify its NOT a home setup it will be an enterprise server. I have several thousand dollars to spend. Any directions on manufacturers would be great too - as of now I am looking at supermicro. Also it is a direct 10gig NIC.
post #5 of 13
How many minutes of the day are you going to be moving 4gb per second?
    
CPUMotherboardGraphicsRAM
2x intel Xeon E5-2650 Supermicro MBD-X9DR3-F-O Onboard awesomeness 8 x 8GB Kingston DDR3 1333 ECC 
Hard DriveCoolingOSMonitor
4x WD Green 2TB in RAID 10 2x Coolermaster Hyper 212 EVOs Windows Server 2012 Datacenter 3x Dell Ultrasharp U2410s 
PowerCase
Corsair AX1200 Case Labs TX10-D 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
2x intel Xeon E5-2650 Supermicro MBD-X9DR3-F-O Onboard awesomeness 8 x 8GB Kingston DDR3 1333 ECC 
Hard DriveCoolingOSMonitor
4x WD Green 2TB in RAID 10 2x Coolermaster Hyper 212 EVOs Windows Server 2012 Datacenter 3x Dell Ultrasharp U2410s 
PowerCase
Corsair AX1200 Case Labs TX10-D 
  hide details  
Reply
post #6 of 13
Thread Starter 
From 8AM ~ 2 gigabits per second 10-4 will peak ~ 4 but we want this to be a long term solution that can handle up to 10 gigabits per second
post #7 of 13
At 4gbps, you will fill a 1TB array in half an hour. How do you plan on moving this data off the server? How are you going to warehouse it?

Regardless, I really hope you have more than "several thousand" to throw at this. I just invoiced a PE T620 with 10 15k 300GB drives, a PERC 710, dual E5-2650s and 32GB RAM. Cost was over $20,000, granted it did include 5 SQL CALs. That system wouldn't be up to your task.
Edited by Oedipus - 2/21/13 at 8:07pm
    
CPUMotherboardGraphicsRAM
2x intel Xeon E5-2650 Supermicro MBD-X9DR3-F-O Onboard awesomeness 8 x 8GB Kingston DDR3 1333 ECC 
Hard DriveCoolingOSMonitor
4x WD Green 2TB in RAID 10 2x Coolermaster Hyper 212 EVOs Windows Server 2012 Datacenter 3x Dell Ultrasharp U2410s 
PowerCase
Corsair AX1200 Case Labs TX10-D 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
2x intel Xeon E5-2650 Supermicro MBD-X9DR3-F-O Onboard awesomeness 8 x 8GB Kingston DDR3 1333 ECC 
Hard DriveCoolingOSMonitor
4x WD Green 2TB in RAID 10 2x Coolermaster Hyper 212 EVOs Windows Server 2012 Datacenter 3x Dell Ultrasharp U2410s 
PowerCase
Corsair AX1200 Case Labs TX10-D 
  hide details  
Reply
post #8 of 13
Yeah.... you will probably want a huge RAM write buffer.... then maybe a RAID10 SSD write buffer.... and then large RAID5/10 of HDDs.

You're looking at $15K+ easy for enterprise hardware.
Edited by DuckieHo - 2/21/13 at 8:20pm
Once again...
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
  hide details  
Reply
Once again...
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
  hide details  
Reply
post #9 of 13
Thread Starter 
Quote:
Originally Posted by Oedipus View Post

At 4gbps, you will fill a 1TB array in half an hour. How do you plan on moving this data off the server? How are you going to warehouse it?

Regardless, I really hope you have more than "several thousand" to throw at this. I just invoiced a PE T620 with 10 15k 300GB drives, a PERC 710, dual E5-2650s and 32GB RAM. Cost was over $20,000, granted it did include 5 SQL CALs. That system wouldn't be up to your task.


Holy crap..

Yeah I didn't actually think about how easily the drives would be filled up. We have an overall budget of 50k but it needs to go into 2 other servers as well.

If I may ask why did you go 15k drives instead of SSD? Cheaper?
post #10 of 13
Quote:
Originally Posted by j3st3r View Post


If I may ask why did you go 15k drives instead of SSD? Cheaper?

Because SSDs will wear out a whole lot faster than 15K SAS drives, especially with the amount of writes you are talking about.
VH001
(17 items)
 
VH002
(15 items)
 
VH003
(16 items)
 
CPUCPUMotherboardGraphics
Intel Xeon E5-2670 Intel Xeon E5-2670 SuperMicro X9DR7-LN4F Onboard Graphics 
RAMHard DriveHard DriveHard Drive
128GB (16 x 8GB) 1333MHz ECC Registered DDR3 4 x Micron M500 480GB SSD 4 x Hitachi 450GB 10K SAS 8 x Toshiba 5TB 
Hard DriveCoolingCoolingOS
Intel 750 400GB NVME SSD 2 x Dynatron R22 6 x SuperMicro 5K RPM Fans VMWare vSphere 6 
MonitorPowerCaseOther
IPMI SuperMicro 650W PSU SuperMicro CSE-833T-650B Black 3U Rackmount Ser... Intel X520-T2 Dual Port 10GbE Server Adapter 
Other
HP P420 1GB Smart Array Controller 
CPUCPUMotherboardGraphics
Intel Xeon E5-2670 Intel Xeon E5-2670 Intel S2600CP2J Onboard Graphics 
RAMHard DriveHard DriveHard Drive
128GB (16 x 8GB) 1333MHz ECC Registered DDR3 4 x Micron M500 480GB SSD 4 x Hitachi 450GB 10K SAS Intel 750 400GB NVME SSD 
CoolingOSMonitorPower
2 x Intel AUPSRCBTP VMWare vSphere 6 IPMI Intel 750W PSU 
CaseOtherOther
Intel P4216XXMHGR 4U Server Chassis Intel X520-T2 Dual Port 10GbE Server Adapter HP P420 1GB Smart Array Controller 
CPUCPUMotherboardGraphics
Intel Xeon E5-2670 Intel Xeon E5-2670 SuperMicro X9DR3-F Onboard Graphics 
RAMHard DriveHard DriveHard Drive
128GB (16 x 8GB) 1333MHz ECC Registered DDR3 4 x Micron M500 480GB SSD 4 x Hitachi 450GB 10K SAS Intel 750 400GB NVME SSD 
CoolingCoolingOSMonitor
2 x Dynatron R22 3 x SuperMicro 5K RPM Fans VMWare vSphere 6 IPMI 
PowerCaseOtherOther
SuperMicro 563W PSU SuperMicro CSE-825TQ-563LPB 2U Rackmount Server... Intel X520-T2 Dual Port 10GbE Server Adapter HP P420 1GB Smart Array Controller 
  hide details  
Reply
VH001
(17 items)
 
VH002
(15 items)
 
VH003
(16 items)
 
CPUCPUMotherboardGraphics
Intel Xeon E5-2670 Intel Xeon E5-2670 SuperMicro X9DR7-LN4F Onboard Graphics 
RAMHard DriveHard DriveHard Drive
128GB (16 x 8GB) 1333MHz ECC Registered DDR3 4 x Micron M500 480GB SSD 4 x Hitachi 450GB 10K SAS 8 x Toshiba 5TB 
Hard DriveCoolingCoolingOS
Intel 750 400GB NVME SSD 2 x Dynatron R22 6 x SuperMicro 5K RPM Fans VMWare vSphere 6 
MonitorPowerCaseOther
IPMI SuperMicro 650W PSU SuperMicro CSE-833T-650B Black 3U Rackmount Ser... Intel X520-T2 Dual Port 10GbE Server Adapter 
Other
HP P420 1GB Smart Array Controller 
CPUCPUMotherboardGraphics
Intel Xeon E5-2670 Intel Xeon E5-2670 Intel S2600CP2J Onboard Graphics 
RAMHard DriveHard DriveHard Drive
128GB (16 x 8GB) 1333MHz ECC Registered DDR3 4 x Micron M500 480GB SSD 4 x Hitachi 450GB 10K SAS Intel 750 400GB NVME SSD 
CoolingOSMonitorPower
2 x Intel AUPSRCBTP VMWare vSphere 6 IPMI Intel 750W PSU 
CaseOtherOther
Intel P4216XXMHGR 4U Server Chassis Intel X520-T2 Dual Port 10GbE Server Adapter HP P420 1GB Smart Array Controller 
CPUCPUMotherboardGraphics
Intel Xeon E5-2670 Intel Xeon E5-2670 SuperMicro X9DR3-F Onboard Graphics 
RAMHard DriveHard DriveHard Drive
128GB (16 x 8GB) 1333MHz ECC Registered DDR3 4 x Micron M500 480GB SSD 4 x Hitachi 450GB 10K SAS Intel 750 400GB NVME SSD 
CoolingCoolingOSMonitor
2 x Dynatron R22 3 x SuperMicro 5K RPM Fans VMWare vSphere 6 IPMI 
PowerCaseOtherOther
SuperMicro 563W PSU SuperMicro CSE-825TQ-563LPB 2U Rackmount Server... Intel X520-T2 Dual Port 10GbE Server Adapter HP P420 1GB Smart Array Controller 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Servers
Overclock.net › Forums › Specialty Builds › Servers › Question on massive I/O on a server