[King_Yoshi]

I am looking for a good, silent, home server that can run Vmware 6.5 24/7.
I currently have an old Supermicro 1U Server X7DBU 2x Xeon L5420 2.5ghz Quad Core 32gb server. (I bought it used for $100)
The thing is a jet engine. I currently wear PELTOR H10A Optime 105 Over-the-Head Earmuffs when working with it and cannot let it run when I am not home due to the noise.

It needs to have at least 32 gigs of RAM and have a processor that can handle running multiple Windows Server VMs at the same time.
I am using it to set up a at-home network lab so I can study for the VMWare VCP-DCV, Microsoft Windows Server 2012/2016, and CCNA certifications.

I will then connect it to the internet so that I can access it on the road.

My budget is around $300-$400, therefore I understand that I most likely would need to buy 2nd hand equipment at this price point.

Any and all suggestions are greatly appreciated.

 

[thymedtd]

Currently I'm running a dual 1366 setup with great results. Had it running esxi now im running unraid but should satisfy your needs. Parts I'm using as a general guide.
-Tyan S7012 board (available on the bay for $60 to $80)
-a pair of Xeon E5649 (6 core 12 thread 2.53 ghz each ~$20 a piece also on the bay)
-12x4GB DDR3 ECC ram (rams gone up significantly since i purchase i got my 6x4gb kits for about $30 a piece now its $60 for the same.
-2x Hyper 212 evos ($30 a piece but definitely quieter than server coolers)
-Rosewill RSV 4000 or similar amazon ~$80
-EVGA 1300 watt gold psu (extremely overkill a 650w or 750w is enough depending on other devices)
-some sata cards and gpus for storage and vm passthrough

That tyan board actually has 18 ram slots and with higher capacity dimms supports up to 288GB, you could have a similar setup for about $320 plus whatever you get for a psu. Also something nice is you can run the board in a single socket config so you only need to get 1 cpu now and can always upgrade/get a second down the road. I would recommend at least 24GB or ram to start with though as it goes pretty fast once its gone. BUt again that can be added later on (with 4gb dimms you'll max out at 72GB in this board.
Currently I'm waiting on the arrival of some X5680's to drop into Eclipse (in sig) to gain single core performance since i also use it to stream games from around my house. I'll add that the 120mm fans that come with the case arent the best, they plug into molex and have no adjustment, not crazy loud out of the gate but if you want it quiet id recommend a fan hub and a pack of these at $20 for 5 theyll do the job and not b crazy loud, if your looking at true silence it doesn't exist in the pc world, moving things make noise, be it a water pump, fan, or coil whine, noise will rear its ugly head. That being said Noctua is generally considered "quiet enough" but at $20 a fan that gets expensive, but again its something that could be upgrades in time. LGA 1366 is still a pretty viable platform for most, sure its not cutting edge but if you need cores and ram for vms its probably the most cost effective way of getting it. Good luck on the server, i know once i started working with server grade hardware theres no going back.

 

[King_Yoshi]

Hello thymedtd,

Thank you for very thorough response.
Quote:

Originally Posted by thymedtd

-Tyan S7012 board (available on the bay for $60 to $80)

That tyan board actually has 18 ram slots and with higher capacity dimms supports up to 288GB, you could have a similar setup for about $320 plus whatever you get for a psu. Also something nice is you can run the board in a single socket config so you only need to get 1 cpu now and can always upgrade/get a second down the road.

That looks like a good board. I will definitely look into it.
I will definitely be getting 2 CPUS for it.
Quote:

Originally Posted by thymedtd

-Rosewill RSV 4000 or similar amazon ~$80

How quiet is this case? Would I be able to sit in the same room as it without being bothered by the noise
This is my main concern.
Quote:

Originally Posted by thymedtd

I would recommend at least 24GB or ram to start with though as it goes pretty fast once its gone.

My current server has 32 gigs of RAM so I will be getting a minimum of 32 gigs, if not more.
I will have to see what RAM is compatible with this board.
Quote:

Originally Posted by thymedtd

That being said Noctua is generally considered "quiet enough" but at $20 a fan that gets expensive, but again its something that could be upgrades in time.

If it makes things quiet, then I will get them.

Regarding fans, I will most likely be getting some of these;
Scythe GentleTyphoon D1225C12B4AP-14 120mm 1450 rpm Silent Case Fan

They are a bit pricy, but some of the quietest fans I have ever owned.
I use them in my my current computer.
Quote:

Originally Posted by thymedtd

...if your looking at true silence it doesn't exist in the pc world....

Actually, this is starting to change.

Take a look at this new (Very expensive) fanless case that will be coming out.
Calyos FANLESS PC / Workstation

They have a kickstarter for it.

 

[thymedtd]

I was actually thinking about that case when i posted it, I saw the LTT video a while back. Thats as close as we'll get to silence i think though, and even then if you get the wrong psu you may start to hear coil whine, or even from your gpu. With that said, the server case from Rosewill can be quiet with the right fans and heat sinks. I'm still running the stock fans since my server is in a rack in a separate room of my house i dont need to change them. Some gentle typhoons would work wonders on it for volume. I can let you know how big of a difference my new heatsinks make, waiting on some hyper 212 evos to replace stock intel 80watt tdp. Those should be here later today and im sure even that small upgrade will quiet it down a lot. As far as gentle typhoon or noctua, i think thats still a debate as to which one is "best". If they're quiet enough for you now though then i wouldn't tell you to change it. I think its mostly personal preference at this point but i could be wrong. The Tyan motherboard runns DDR3 memory, ive put both non ECC and registered ECC dimms with no problems. heres a link to the tyan page about the board, should answer questions about memory compatibility. Always glad to help as well, I'll try and keep up on your work and hopefully its smooth sailing for you

 

[King_Yoshi]

Quote:

-some sata cards and gpus for storage and vm passthrough

This is something I was very interested in doing.
How well is the GPU passthrough working? I know that Vmware is a bit "picky" when it comes to which types of videocards work with passthrough.

I saw you were using a "EVGA 960 SSC 4GB". Is this the only videocard you could get working with it? Is there a specific reason you chose this card?

 

[thymedtd]

GPU pass through was part of the reason i switched to unraid. VMware works much easier with AMD cards because they don't lock down the driver as much. At the time of me starting my server I only had a HD 6670, a pair of GTX 460 SE cards and a single GTX 960 SSC. Tried for a while to get them working with vmware with no luck, switched to unraid made a few tweaks and had it semi working with amd first then nvidia. This is when i found out my 960 was having an issue, did an RMA on the card and while i was waiting on that i got a Radeon 7870 to game with and test. Now i have unraid running with both of those cards and a separate but very similar box (same motherboard xeon e5649 cpus and 12 gb of ram instead of 48) that has esxi which i'll be doing some testing with. In fact if i was building a new box for gpu pass through id try to go with AMD cards since they've worked more seamlessly for me so far. The 960 was just what i had at the time. Also i did receive the hyper 212 evos and had an issue mounting them to the tyan board. it had an attached backplate so it doesnt support that 1366 mounting out of the box for server boards which i found odd. it took some playing with and modification but i got them attached. there is about 2-3mm of clearance issue to the top of the case but not a huge issue since it still closes and i have room in my rack so its not a tight fit. Whats the plan for VMware in particular? Is it workload specific or just what you know? As i was saying if GPU pass through is important i may be inclined to recommend another option, unless you'll be going with AMD cards then it shouldnt make much of a difference for compatability.

 

[King_Yoshi]

Quote:

Originally Posted by thymedtd

Whats the plan for VMware in particular? Is it workload specific or just what you know? As i was saying if GPU pass through is important i may be inclined to recommend another option, unless you'll be going with AMD cards then it shouldnt make much of a difference for compatability.

I will be studying for a couple certifications, including my Vmware specific VCP-DCV (VMware Certified Professional - Data Center Virtualization).
I am aiming to run 2 Microsoft Servers, as well as a number of Windows and Linux boxes to simulate a business environment. Simultaneously I will be connecting this virtualized network to 5 physical switches 2-4 routers and a firewall, which I will be using for my CCNA.

Again, my goal is to create as realistic of a business environment as I can.
If you are wondering why I chose Vmware over the other virtualization options, its because it has proven to be one of the most reliable and commonly seen solutions in business environments.
I am hoping to use the skills I have gained from this and use it to apply for similar positions.

Regarding the videocard passthrough. It's a personal interest of mine.
I have always wanted top be able to run some more graphically intensive applications on VMs but have found that most virtualization solutions do not work well or even allow it.

 

[thymedtd]

Definitely a different workload than myself. I'd recommend at least 48GB of ram, i am generally anywhere between half and 3/4 ram usage in my server and from the sounds of it you'll be running more vm's than myself. It's fairly easy to over commit cpu resources, ram is a whole different story though, ram always goes quick in virtual environments. I also can't argue with vmware as a choice, its stability and functional use is what you'd wan't to use if your getting into that industry. If your curious how something(s) might run on the hardware i can spin up a couple vms and do some testing for you before you pull the trigger on the hardware. All that being said, in your price range i don't think you could go wrong with a second hand dual socket lga1366. I might also recommend grabbing an extra nic or 2 for playing with physical pass through.

 

[King_Yoshi]

I just ordered the motherboard and case. I will be slowly gathering the part s to build this server.

The only challenge I have had so far is finding compatible RAM.
I looked at the specs on Tyan's website but have not had much luck finding compatible RAM kits.

 

[King_Yoshi]

I just found this RAM kit.
http://www.memory4less.com/kingston-12gb-ddr3-pc8500-kvr1066d3d4r7sk3-12g

Looks a bit pricy, but I believe its compatible.

 

[thymedtd]

definitely pricey. I've had good luck with this ram in my setup, generally ive had good luck with ram certified for dell workstations.
https://www.ebay.com/itm/24GB-6X4GB...482783?hash=item25897d56df:g:yUAAAOxyRhBS9mPN

 

[King_Yoshi]

Does anyone have any thoughts on this RAM kit for the above server?

72GB KIT (9 x 8GB) For Tyan S7000 Series S7012 (S7012GM4NR) (ECC Registered) S7012 (S7012WGM4NR) (ECC Registered) S7016 (S7016GM3NR) (ECC Registered) S7016 (S7016WGM3NR) (ECC Registered). DIMM DDR3 ECC Registered PC3-12800 1600MHz Dual Rank RAM Memory. Genuine A-Tech Brand.
https://www.amazon.com/S7012GM4NR-R...id=1519173930&sr=8-1&keywords=TYAN+S7012GM4NR

I also found this RAM kit, it seems to be Much cheeper then the above ones. It seems another user had used it with theur Tyan.



https://www.ebay.com/itm/24GB-6X4GB...=item25897d56df:g:yUAAAOxyRhBS9mPN&rmvSB=true
http://rover.ebay.com/rover/1/711-5...482783?hash=item25897d56df:g:yUAAAOxyRhBS9mPN

 

[King_Yoshi]

definitely pricey. I've had good luck with this ram in my setup, generally ive had good luck with ram certified for dell workstations.
https://www.ebay.com/itm/24GB-6X4GB...482783?hash=item25897d56df:g:yUAAAOxyRhBS9mPN

Ahh that is the exact one I was just looking at. I may buy two of them.

 

[King_Yoshi]

I was looking to use these as my coolers: https://www.newegg.com/Product/Product.aspx?Item=N82E16835186066

But I was reading that there have been various mounting difficulties with this motherboard and various coolers.
is there anything I should lookout for? I read through your build log. Any suggestions regarding coolers?
(I will be going with Intel Xeon Processor X5690 3.46GHz/6-core/12MB/130W processors.)

 

[King_Yoshi]

Regarding the coolers. I think I have decided to go with these beastly Noctua's.

NH-U12DX i4
https://noctua.at/en/nh-u12dx-i4
https://www.amazon.com/Noctua-Coole...F8&qid=1519669113&sr=8-2&keywords=NH-U12DX+i4

 

[thymedtd]

yup those should be perfect. the mounting system is what you want for that motherboard since it doesnt need stand offs just the screws to go into the backplate. 92mm fan should give you plenty of clearance in the rosewill case. im just a bit to tall with 120's so that setup should be perfect; quiet and cool.

 

[King_Yoshi]

AMI BIOS Post Code "10"

Thank you for all your help so far.

I ran into a problem during my build.

When trying to boot with a new X5690 CPU I keep getting AMI BIOS Post Code "10".
After looking it up I found it referred to the following.

"CMOS shutdown register read/write error
The shutdown for the CMOS has failed"
Source: http://www.bioscentral.com/beepcodes/amibeep.htm

It is my understanding this indicates a problem with the motherboard itself, or a incompatibility with a piece of hardware.
(I could be wrong, but this is what I gathered from my research.)

I tested each piece of hardware, one by one and found that I only receive this post code when I have a X5690 CPU installed.
(For example I just inserted the RAM and powered the server on and received no error. I repeated these steps for all other hardware.)

Would this mean that the CPUs are bad? Or that there is a problem with the board.

I may need to purchase another CPU for testing. (But I really do not want to spend more money as these were new CPUs to begin with.)

Below are links to the ebay listings I bought the board and CPUs from. (I ended up spending more then I originally budgeted, since I bought parts that were listed as "New".)

https://www.ebay.com/itm/Intel-Xeon...e=STRK:MEBIDX:IT&_trksid=p2057872.m2749.l2649
https://www.ebay.com/itm/NEW-Tyan-S...e=STRK:MEBIDX:IT&_trksid=p2057872.m2749.l2649

 

[King_Yoshi]

I believe I discovered the problem. In the fine print on the motherboard it says that the Bios needs to be updated in order for it to be compatible with the 5600 Series Xeon processors.
http://oi64.tinypic.com/2ylmxlh.jpg

This is the Tyan docs page it appears on.
http://www.tyan.com/doc/support_lists/CPU/LGA1366_MB

So I went ahead and purchased an older, cheap pair of processors so I can update the bios. (I bought 2 just incase 1 was bad.)

It was a total of $14 for two of them so not a big deal.

https://www.ebay.com/itm/Matched-Pa...076296&hash=item2a9f97f5af:g:-noAAOSwZDBZ1ZOk

 

[King_Yoshi]

No luck so far.

With the E5540 Processors I get AMI BIOS Post Code "04".
I have tried resetting the bios using the jumper, but still have had no luck.

I am beginning to think that this board is bad.

I have now tried the board with 4 different Intel E5540 Processors and 2 different Intel X5690s. With the E5540 processors I get Post Code 04, which usually indicates a problem with the bios. I have tried using the processors one at a time in each slot as well as multiple processors in both slots. I even tried with just the processors installed and no RAM or other devices connected.
When I tried to use the X5690s I received post code 10, which usually indicates a damaged motherboard or a problem with a outdated Bios. (I had wanted to use a E5540 to update the bios, but it will not get past pos.)

The rest of the build's specifications are below.

Processors Tried
Intel Xeon X5690
Intel Xeon CPU E5540

RAM
Crucial CT51272bb1339 4GB DDR3 ECC Registered Kit (PC3-1333, DDR3-1333MHz)

Power Supply
Corsair HX1000i

Hardrive
None Connected

 

[King_Yoshi]

I just had the motherboard replaced with a new one that the seller themselves tested.
I am still getting Error 10..

I am now a complete loss of what to do, as this makes no sense...

I had even troubleshot the last board with Tyan who said it was bad.