[j3st3r]

Hey I am looking at servers for one of our departments at work. The machine will essentially store logs at around 4 gbits a second but we are looking for a long term solution that could go up to 10 gigabits per second. We have narrowed down most of the parts we need, but in terms of HDDs we are still unsure what could be the bottleneck.

I was thinking of either doing a raid array or introducing an SSD that would get backed up every night onto a regular spin plate 7200. I am not sure which is more efficient. Any input on storing large amounts of data on the fly would be great. These are going to be standard PCAP files but they will be of an entire network so the throughput will be enormous. Cheers.

 

[herkalurk]

So essentially you're packet capturing your network? Just to clarify.....

There are products for that. Extrahop being the first that comes to mind, really cool stuff, but to be honest if you have 4 gbit a second coming in, don't know why you're trying a home built solution that has no support or guarantees...

 

[Oedipus]

I never recommend homebuilt machines of any kind, let alone servers, in a business environment. Homebuilding a machine that processes data that quickly would be suicidal.

Anyway... how's your switching? Is this going to be FC or 10gbe, or some kind of 4x1gb setup?

 

[j3st3r]

Just to clarify its NOT a home setup it will be an enterprise server. I have several thousand dollars to spend. Any directions on manufacturers would be great too - as of now I am looking at supermicro. Also it is a direct 10gig NIC.

 

[Oedipus]

How many minutes of the day are you going to be moving 4gb per second?

 

[j3st3r]

From 8AM ~ 2 gigabits per second 10-4 will peak ~ 4 but we want this to be a long term solution that can handle up to 10 gigabits per second

 

[Oedipus]

At 4gbps, you will fill a 1TB array in half an hour. How do you plan on moving this data off the server? How are you going to warehouse it?

Regardless, I really hope you have more than "several thousand" to throw at this. I just invoiced a PE T620 with 10 15k 300GB drives, a PERC 710, dual E5-2650s and 32GB RAM. Cost was over $20,000, granted it did include 5 SQL CALs. That system wouldn't be up to your task.

 

[DuckieHo]

Yeah.... you will probably want a huge RAM write buffer.... then maybe a RAID10 SSD write buffer.... and then large RAID5/10 of HDDs.

You're looking at $15K+ easy for enterprise hardware.

 

[j3st3r]

Quote:

Originally Posted by Oedipus

At 4gbps, you will fill a 1TB array in half an hour. How do you plan on moving this data off the server? How are you going to warehouse it?

Regardless, I really hope you have more than "several thousand" to throw at this. I just invoiced a PE T620 with 10 15k 300GB drives, a PERC 710, dual E5-2650s and 32GB RAM. Cost was over $20,000, granted it did include 5 SQL CALs. That system wouldn't be up to your task.

Holy crap..

Yeah I didn't actually think about how easily the drives would be filled up. We have an overall budget of 50k but it needs to go into 2 other servers as well.

If I may ask why did you go 15k drives instead of SSD? Cheaper?

 

[jibesh]

Quote:

Originally Posted by j3st3r

If I may ask why did you go 15k drives instead of SSD? Cheaper?

Because SSDs will wear out a whole lot faster than 15K SAS drives, especially with the amount of writes you are talking about.

 

[Oedipus]

Quote:

Originally Posted by j3st3r

Holy crap..

Yeah I didn't actually think about how easily the drives would be filled up. We have an overall budget of 50k but it needs to go into 2 other servers as well.

If I may ask why did you go 15k drives instead of SSD? Cheaper?

Primarily. Dell's "value" MLC drives are like $450 each for 100GB.

I ran some checks on a similar system (8 drives, though) and it was pulling like 600MBps reads and 500MBps writes or something like that. Not even half where you need to be if you're looking at 10gb.

 

[j3st3r]

Quote:

Originally Posted by Oedipus

Primarily. Dell's "value" MLC drives are like $450 each for 100GB.

I ran some checks on a similar system (8 drives, though) and it was pulling like 600MBps reads and 500MBps writes or something like that. Not even half where you need to be if you're looking at 10gb.

Damnit.. lol I am not sure how I am going to pull this off then.

 

[herkalurk]

I'm still going to point out, you should look at something like Extrahop......Awesome stuff IMO. My company couldn't afford it, but it sounds like your company has some budget money to spend. We we're looking at a box that could analyze 3 Gbit/sec and it wasn't more than 40K ( i think...)