[Official] ASUS X99-E WS Owner's thread

MasterCyclone3D · Oct 14, 2014

Quote:

Originally Posted by M125

Looks like you can pick up a tray Xeon E5-1660 v3 eight-core here now: https://www.superbiiz.com/detail.php?name=E5-1660V3*
The boxed Xeon E5-1650 v3 six-core is also listed on Superbiiz's site: https://www.superbiiz.com/detail.php?name=E51650V3BX*

Both are unlocked (to my knowledge and experience) and comparable in price to the 5960X and 5930K, respectively. They should clock comparable to the i7 processors in hands more competent than my own.

Pair a Xeon with some ECC DDR4 listed on the QVL, found here: https://www.superbiiz.com/detail.php?name=D4218G4S1

With those parts, you'll make a real workstation out of the X99-E WS.

The E5-1660 v3 is still not available at my corporate vendor. I guess I'll have to wait until 14nm Broadwell-E lands to move to an eight-core. Six 4.0 Ghz Haswell cores are much better than eight 2.13 Ghz Nehalem cores, so I'll see an improvement in license utilization with a Xeon E5-1650 v3, even if it is missing two cores.

*I swear I do not work for Superbiiz! They happen to stock a lot of server-grade components.

I wish they made the ECC memory in Black PCB. Would be cool if some company do this.

Quote:

Originally Posted by FreeElectron

Did any of you try to watercool this mobo?
Any advices on that?

If I did get watter cooling THen it would have been 2x 220mm Rad and 2x140mm Rad and get some little 1/8 hard aluminum plates and cut out the size for the mosfet and the other side those chips and mount a water block for VRM mosfet cooler and don't forget to add a heat transfer plates for the back of the motherboard here link http://koolance.com/mvr-100-motherboard-vreg-water-block
http://koolance.com/index.php?route=product/category&path=29_148_108
And I would get a chipset cooler http://koolance.com/cold-plate-40mm-plt-un40f
And then a cpu cooler http://koolance.com/cpu-380i-processor-water-block
And then cut out some plates 1/16 thick hard aluminum plates and add them to the memory rams and add memory water block http://koolance.com/index.php?route=product/category&path=29_148_56
And then add two video card water blocks http://koolance.com/video-card-vga-nvidia-geforce-gtx-titan-water-block-vid-nxttn2
And double my pump one in and one going out http://koolance.com/pmp-500-pump-g-1-4-bsp

This is what I would do if I had the money for water cooling system for the Rosewill blackHawk ultra case.

adnadn · Oct 14, 2014

Quote:

Originally Posted by Margammor

I use two Samsung XP941 M.2 drives on the X99 DeLuxe and no issues. One is a boot drive nd the other a Photoshop and Premiere scratch disk. Chrystal Diskmark read and write 1094/1016 MB/s. Had some issues getting it to work as a boot drive, in W8.1. Doe not format it and W8.1 automatically makes it a GPT bootdrive.

This is what I am afraid of. I am still with Win7(64) with several TBs installed on the old rig. It would be good to just change the hardware and do P&P but this is not feasible as we all know. I need to re install all my stuff - will take me a month or more I guess. I did not want to move to 8.1 as I have a lot of hardware specific drivers and I do not know how they would work in 8.1. Thus, the full rationale for moving to x99 fully capable with 4xSLI 16x was to include the system installed on M.2 drive. Any hint/step by step advice how to proceed?

Margammor · Oct 14, 2014

Quote:

Originally Posted by adnadn

This is what I am afraid of. I am still with Win7(64) with several TBs installed on the old rig. It would be good to just change the hardware and do P&P but this is not feasible as we all know. I need to re install all my stuff - will take me a month or more I guess. I did not want to move to 8.1 as I have a lot of hardware specific drivers and I do not know how they would work in 8.1. Thus, the full rationale for moving to x99 fully capable with 4xSLI 16x was to include the system installed on M.2 drive. Any hint/step by step advice how to proceed?

I did a new rig minimal setup with one M.2 drive and a fresh install of W8.1. Got it all updated with the latest drivers, bios and W8.1 updates and all. OC'ed it and made it stable. Then added other HD's and cards and so one. Made them work and all and installed all other software I needed. Transferred data from the old rig to the new one and done. It is a pain in the ... project, but a clean install gets a lot of old issues away and a fast clean system.

Canis-X · Oct 14, 2014

Got back from vacation late on Sunday, so I began work on Monday afternoon, got the board prepped for phase and most of the wiring done so far...

FreeElectron · Oct 14, 2014

Quote:

Originally Posted by Canis-X

Got back from vacation late on Sunday, so I began work on Monday afternoon, got the board prepped for phase and most of the wiring done so far...

Keep us updated

M125 · Oct 14, 2014

Quote:

Originally Posted by FreeElectron

What is the difference between the i7 extreme and the xeon?
Why should someone consider one over the other?

To my knowledge, the Xeon E5-1660 v3 and the i7-5960x are widely comparable and based on the same silicon (could come from the same wafer even) save for a few differences:

Code:

Code:

                C/T  CACHE CLOCK  BOOST  UNLOCKED MSRP(TRAY) VT-x VT-d vPro TXT RDIMM MEMMAX   ECC TCASE 
i7-5960X        8/16 20MB  3.0GHz 3.5GHz YES      $999.00    YES  YES  NO   NO  NO    64/128GB NO  66.8C
Xeon E5-1660 v3 8/16 20MB  3.0GHz 3.5GHz YES      $1080.00   YES  YES  YES  YES YES   768GB    YES 65.9C

Instructions sets supported should be the same across both processors. The Xeon can still use normal UDIMM/non-ECC memory as the i7-5960X would and nine times out of ten would be compatible with the same consumer boards the 5960X is.

Many of us would choose the Xeon over the i7 if only for ECC support. The fact that the Xeon E5-1660 v3 is unlocked is just icing on the cake, so to say. Overclocking and ECC/stability/correctness of output usually do not go in the same sentence, even if you give your CPU enough volts. I find a 24+ hour stable clock/voltage at 4.2 Ghz and back it down to 4.0 GHz with the same voltage to assure stability, and even then I cannot 100% guarantee the same operation as Intel does at factory frequencies and voltages.

Past anything ECC or RDIMM related, the Xeon E5-1660 v3 would be virtually indistinguishable from 5960X in normal OCN-type use. I would think they would overclock the same, but have yet to see widespread results on the E5-1660 v3.

I figure if someone is looking at the X99-E WS for a workstation, they might as well have a Xeon/ECC. Else, if you are not running four GPUs/etc, save your money and get a cheaper board and the 5960X. The PLX chip, while nice to have, is at the bottom of my wishlist. I'm sure we will see this board running four way SLI at x16 on all cards soon enough, though.

Quote:

Originally Posted by MasterCyclone3D

I wish they made the ECC memory in Black PCB. Would be cool if some company do this.

I have never seen black, but I have seen blue PCB ECC in some HP ProLiants and Mac Pros. Every Dell I've worked with has been various shades of green. DDR2-era was the last time I remember seeing black heat-sinks on green PCBs. Seeing a black PCB ECC DIMM ala the low profile Samsung "30nm" DDR3 would be pretty cool.

FreeElectron · Oct 14, 2014

Quote:

Code:
Originally Posted by M125

To my knowledge, the Xeon E5-1660 v3 and the i7-5960x are widely comparable and based on the same silicon (could come from the same wafer even) save for a few differences:

Code:

Code:

C/T CACHE CLOCK BOOST UNLOCKED MSRP(TRAY) VT-x VT-d vPro TXT RDIMM MEMMAX ECC TCASE i7-5960X 8/16 20MB 3.0GHz 3.5GHz YES $999.00 YES YES NO NO NO 64/128GB NO 66.8C Xeon E5-1660 v3 8/16 20MB 3.0GHz 3.5GHz YES $1080.00 YES YES YES YES YES 768GB YES 65.9C

Instructions sets supported should be the same across both processors. The Xeon can still use normal UDIMM/non-ECC memory as the i7-5960X would and nine times out of ten would be compatible with the same consumer boards the 5960X is.

Many of us would choose the Xeon over the i7 if only for ECC support. The fact that the Xeon E5-1660 v3 is unlocked is just icing on the cake, so to say. Overclocking and ECC/stability/correctness of output usually do not go in the same sentence, even if you give your CPU enough volts. I find a 24+ hour stable clock/voltage at 4.2 Ghz and back it down to 4.0 GHz with the same voltage to assure stability, and even then I cannot 100% guarantee the same operation as Intel does at factory frequencies and voltages.

Past anything ECC or RDIMM related, the Xeon E5-1660 v3 would be virtually indistinguishable from 5960X in normal OCN-type use. I would think they would overclock the same, but have yet to see widespread results on the E5-1660 v3.

I figure if someone is looking at the X99-E WS for a workstation, they might as well have a Xeon/ECC. Else, if you are not running four GPUs/etc, save your money and get a cheaper board and the 5960X. The PLX chip, while nice to have, is at the bottom of my wishlist. I'm sure we will see this board running four way SLI at x16 on all cards soon enough, though.
I have never seen black, but I have seen blue PCB ECC in some HP ProLiants and Mac Pros. Every Dell I've worked with has been various shades of green. DDR2-era was the last time I remember seeing black heat-sinks on green PCBs. Seeing a black PCB ECC DIMM ala the low profile Samsung "30nm" DDR3 would be pretty cool.

You got me interested.
I initially chose this board because of the pcie management including the plx chip.
The R5E and the Deluxe have some shared connections SATA Express+ M.2. + PCIE + USB and i wanted to avoid that
Also the ability to install PCIE cards anywhere i want to avoid sandwiching graphics cards on air.

But now i am interested about the xeon
I initially searched to check if the xeons are overclock-able or not and found at that they are not overclockable (through multiplier) so i discarded the xeon option.

But now i may consider it again.

As a developer what does the Registered memory and the ECC features add?

I intend to play and develop and test on that rig.

Also will this board support ECC and Registered Memory?
Finally which intel xeon v3 cpus are unlocked and which ones are not?

ocmcdizzle · Oct 14, 2014

Quote:

Originally Posted by adnadn

This is what I am afraid of. I am still with Win7(64) with several TBs installed on the old rig. It would be good to just change the hardware and do P&P but this is not feasible as we all know. I need to re install all my stuff - will take me a month or more I guess. I did not want to move to 8.1 as I have a lot of hardware specific drivers and I do not know how they would work in 8.1. Thus, the full rationale for moving to x99 fully capable with 4xSLI 16x was to include the system installed on M.2 drive. Any hint/step by step advice how to proceed?

I installed the XP941 as a boot drive last night. The trick was to use a program called rufus to burn the ISO, and set it to a GPT partition instead of MBR (universal usb installer didnt have this option afaik). I used NTFS but I think FAT works as well. Then just hit F8 during post and boot from the UEFI version of the usb drive!

M125 · Oct 15, 2014

Quote:

Originally Posted by FreeElectron

As a developer what does the Registered memory and the ECC features add?

Registered memory adds no value on this platform beyond maybe allowing more than 128GB of DDR4 down the road. This is a departure from the previous P9X79-E WS that required UDIMMs as far as I know. Historically UDIMM ECC surpassed RDIMM ECC in cost, so this could be a great change.

I do finite element analysis on forging dies, and study how different designs impact material flow on a micro and macro scale. If a single point vector in a mesh of say 300K elements is computed erroneously (its step position/acceleration change is too large or small or even in the wrong direction), it can seriously impact our findings, or even crash the whole simulation. ECC can correct one bit flop per word, which, considering the (in)frequency of bit errors, is plenty. Your chances of a bit error are increased the more RAM you allocate. The software I use likes to run about 4GB per core with our settings. Chances are that without ECC, you will occasionally be fooled into thinking a bad result can be put into production

If you do any scientific computing that is memory intensive, or work that requires absolute precision in output (designs that people trust their life and well being with), you would be better off with ECC.
Quote:

Originally Posted by FreeElectron

Also will this board support ECC and Registered Memory?

See this page: http://www.asus.com/us/Commercial_Servers_Workstations/X99E_WS/HelpDesk_QVL/
Specifically: http://dlcdnet.asus.com/pub/ASUS/server/X99-E_WS/QVL/X99-E_WS_MEMORY_DDR4_ECC_RDIMM_QVL_140917.pdf

It looks like Samsung and Crucial have both certified the X99-E WS for use with a model each of their ECC DDR4 RDIMMs.
Quote:

Originally Posted by FreeElectron

Finally which intel xeon v3 cpus are unlocked and which ones are not?

This is something us consumers have to figure out by trial and error.

I have found that the (tray) Xeon E5-1650 v3 and (tray) Xeon E5-1660 v3 are both unlocked.

I'll try to keep this list up-to date when I find any more information or actually test any other Xeon E5-16XX v3s.

TheGovernment · Oct 15, 2014

When I installed win 8.1 on my xp941, I did nothing but install the drive and put my windows disk in. Went straight to setup, formatted the drive and was done. I didn't have to do any of the stuff some of the other did. My issue was booting from the drive but once I found that guide I posted, it worked great.

FreeElectron · Oct 15, 2014

Quote:

Originally Posted by M125

Registered memory adds no value on this platform beyond maybe allowing more than 128GB of DDR4 down the road. This is a departure from the previous P9X79-E WS that required UDIMMs as far as I know. Historically UDIMM ECC surpassed RDIMM ECC in cost, so this could be a great change.

I do finite element analysis on forging dies, and study how different designs impact material flow on a micro and macro scale. If a single point vector in a mesh of say 300K elements is computed erroneously (its step position/acceleration change is too large or small or even in the wrong direction), it can seriously impact our findings, or even crash the whole simulation. ECC can correct one bit flop per word, which, considering the (in)frequency of bit errors, is plenty. Your chances of a bit error are increased the more RAM you allocate. The software I use likes to run about 4GB per core with our settings. Chances are that without ECC, you will occasionally be fooled into thinking a bad result can be put into production

If you do any scientific computing that is memory intensive, or work that requires absolute precision in output (designs that people trust their life and well being with), you would be better off with ECC.
See this page: http://www.asus.com/us/Commercial_Servers_Workstations/X99E_WS/HelpDesk_QVL/
Specifically: http://dlcdnet.asus.com/pub/ASUS/server/X99-E_WS/QVL/X99-E_WS_MEMORY_DDR4_ECC_RDIMM_QVL_140917.pdf

It looks like Samsung and Crucial have both certified the X99-E WS for use with a model each of their ECC DDR4 RDIMMs.
This is something us consumers have to figure out by trial and error.

I have found that the (tray) Xeon E5-1650 v3 and (tray) Xeon E5-1660 v3 are both unlocked.

I'll try to keep this list up-to date when I find any more information or actually test any other Xeon E5-16XX v3s.

What do you think about this (Crucial 64GB kit (16GBx4) DDR4 PC4-17000 Registered ECC 1.2V) and Intel® Xeon® Processor E5-1660 v3 (20M Cache, 3.00 GHz)?
Quote:

Originally Posted by Intel® Xeon® Processor E5-1660 v3 (20M Cache, 3.00 GHz)

VID Voltage Range 0.65-1.30V

Does that mean that max voltage is 1.3v or i can add more?
Or is it max stable voltage?

ocmcdizzle · Oct 15, 2014

Quote:

Originally Posted by TheGovernment

When I installed win 8.1 on my xp941, I did nothing but install the drive and put my windows disk in. Went straight to setup, formatted the drive and was done. I didn't have to do any of the stuff some of the other did. My issue was booting from the drive but once I found that guide I posted, it worked great.

Yep, the usb boot disk method is for those who don't go the optical drive / physical windows disc route.

wirk · Oct 15, 2014

I am planning workstation based on the X99-E WS, not at least due to the fact I may need all 7 PCIe slots in the future. I also want a very fast PCIe SSD and I am looking into the new Intel P3600 which is PCIe 3.0x4:

http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-dc-p3600-spec.pdf

This normally would take one PCIe slot but I figured out that it might be possible to use the M.2 slot which is also PCIe 3.0x4 and connect it with the SSD card via suitable PCIe-M.2 adapter which I found here:

http://www.bplus.com.tw/ExtenderBoard/P4SM2.html

Then I could connect the PCIe adapter with the SSD card by flexible ribbon PCIe cable. In this way, all PCIe slots would be free to use and the PCIe SSD storage could still run.

As this is only imagined theoretical solution, could there be any problems here which I can not imagine?

M125 · Oct 15, 2014

Quote:

Originally Posted by FreeElectron

What do you think about this (Crucial 64GB kit (16GBx4) DDR4 PC4-17000 Registered ECC 1.2V) and Intel® Xeon® Processor E5-1660 v3 (20M Cache, 3.00 GHz)?
Does that mean that max voltage is 1.3v or i can add more?
Or is it max stable voltage?

If you want to ensure system stability and proper memory performance with ECC RDIMMs, I would stick with what is listed on the X99-E WS's RDIMM memory QVL. If you need 16GB DIMMs, that means sourcing Crucial MTA36ASF2G72PZ-2G1A2IG exactly. This may be quite difficult to find as of right now.

That VID is just a range of voltages possibly needed for a specific Xeon E5-1660 v3 to run at stock clock frequencies, boost or otherwise. Each CPU will have a unique VID (or multiple ones, depending on boost mode), between the voltages listed on ARK. The upper value is not a voltage limit or max voltage, but the highest voltage you could potentially see a CPU need at stock frequencies, fresh from the factory. I can't find any hard data on Haswell EP, but I would guess 1.4V would be my maximum, where 1.5V could kill it.

He1p1ess n00b · Oct 15, 2014

Guys, I think that Xeon's discussion is offtopic here and would be more appropriate to continue in http://www.overclock.net/t/1510429/so-what-is-the-equivalent-xeon-2011-that-matches-the-5960x

Quote:

Originally Posted by FreeElectron

But now i am interested about the xeon
I initially searched to check if the xeons are overclock-able or not and found at that they are not overclockable (through multiplier) so i discarded the xeon option.

The best 2 sockets Xeons like 10 core E5 2687V3 or 18core 2699V3 are locked, that is why they not considered by overclockers at all
Quote:

Originally Posted by FreeElectron

As a developer what does the Registered memory and the ECC features add?

They add nothing for consumer-grade desktops, except increased latencies and lower speeds. The ECC is only needed on servers or if you desperately need to execute prolonged calculations like F@H, which maybe need a week of 100% CPU load. In that case ECC will correct possible memory errors
Quote:

Originally Posted by FreeElectron

Also will this board support ECC and Registered Memory?

More not than yes, because ECC support is left to Intel C6** server chipset, X99 doesn't have ECC support
Quote:

Originally Posted by FreeElectron

Finally which intel xeon v3 cpus are unlocked and which ones are not?

The 2 sockets E5 26**V3 are all locked, the 1 socket E5 16**V3 are unlocked
But I don't know if E5-16** do support strap overclocking, maybe they don't and only could be overclocked on strap 100 by multi. In that case it is harder to achieve memory speeds higher than 2666MHz

TheGovernment · Oct 15, 2014

Quote:

Originally Posted by ocmcdizzle

Yep, the usb boot disk method is for those who don't go the optical drive / physical windows disc route.

Lol, I missed the part of using usb. Duh. Now it makes sense. I was reading it thinking *** lol

wirk · Oct 15, 2014

Quote:

Originally Posted by He1p1ess n00b

Guys, I think that Xeon's discussion is offtopic here and would be more appropriate to continue in http://www.overclock.net/t/1510429/so-what-is-the-equivalent-xeon-2011-that-matches-the-5960x
The best 2 sockets Xeons like 10 core E5 2687V3 or 18core 2699V3 are locked, that is why they not considered by overclockers at all
They add nothing for consumer-grade desktops, except increased latencies and lower speeds. The ECC is only needed on servers or if you desperately need to execute prolonged calculations like F@H, which maybe need a week of 100% CPU load. In that case ECC will correct possible memory errors
More not than yes, because ECC support is left to Intel C6** server chipset, X99 doesn't have ECC support
The 2 sockets E5 26**V3 are all locked, the 1 socket E5 16**V3 are unlocked
But I don't know if E5-16** do support strap overclocking, maybe they don't and only could be overclocked on strap 100 by multi. In that case it is harder to achieve memory speeds higher than 2666MHz

To clarify: high-end X99 workstation-class motherboards definitely support ECC RDIMM RAM, when running with Xeon processor. This is true for the Asus X99-E WS where somehow clear statement about Xeon support is missing but a list of approved memory cards is available. But for example ASRoCK Extreme11 specs state this directly. For Gigabyte X99-SOC Force there is list of supported Xeon processors, all Xeon's E5 v3 are supported. Regarding overclocking it looks like full overclocking is supported with unlocked Xeon chips, judging from the extreme overclocking tests.

About the use of Xeon E5 1660 v3 vs. i7-5960X one can see it from the point that price difference is small or nonexistent and then why not trying it, especially when heavier loads are to be considered (and/or 128 GB of RAM might be useful).

Canis-X · Oct 15, 2014

Just a bit of an update, nothing really important, but last night I set off to clean my water cooling components. Lot's of boiling water and some soaking in vinegar water solution for the parts that I didn't think should be boiled. I ordered a few parts yesterday from performance PC's since I am changing my water cooling setup a bit. On tonight's agenda, get started on installing the water cooling components. Some small modifications to make to the case, nothing major though. Wish things were going faster, and I'm not looking forward to cleaning up my office after I'm done with the physical portion of the build, but things are coming together nicely I suppose.

FreeElectron · Oct 15, 2014

Quote:

Originally Posted by M125

If you want to ensure system stability and proper memory performance with ECC RDIMMs, I would stick with what is listed on the X99-E WS's RDIMM memory QVL. If you need 16GB DIMMs, that means sourcing Crucial MTA36ASF2G72PZ-2G1A2IG exactly. This may be quite difficult to find as of right now.

That VID is just a range of voltages possibly needed for a specific Xeon E5-1660 v3 to run at stock clock frequencies, boost or otherwise. Each CPU will have a unique VID (or multiple ones, depending on boost mode), between the voltages listed on ARK. The upper value is not a voltage limit or max voltage, but the highest voltage you could potentially see a CPU need at stock frequencies, fresh from the factory. I can't find any hard data on Haswell EP, but I would guess 1.4V would be my maximum, where 1.5V could kill it.

Why this model in particular?
I can't find it in the QVL list.

M125 · Oct 15, 2014

Quote:

Originally Posted by FreeElectron

Why this model in particular?
I can't find it in the QVL list.

From Asus's "X99-E WS memory ECC RDIMM QVL 2014.09.17" here: http://dlcdnet.asus.com/pub/ASUS/server/X99-E_WS/QVL/X99-E_WS_MEMORY_DDR4_ECC_RDIMM_QVL_140917.pdf

Quote:

Samsung M393A1G40DB0-CPB 8GB DDR4-2133 ECC/REG CL15 1 Samsung K4A4G045WD-BCPB IDT 4RCD0124KC0ATG
Micron MTA36ASF2G72PZ-2G1A2IG 16GB DDR4-2133 ECC/REG CL15 2 Micron MT40A1G4HX-093E:A IDT 4RCD0124KC0ATG
Apacer 75.CA3G0.G000B 8GB DDR4-2133 ECC/REG CL15 1 Samsung K4A4G045WD-BCPB IDT 4RCD0124KC0ATG

[Official] ASUS X99-E WS Owner's thread

MasterCyclone3D

adnadn

Margammor

Canis-X

FreeElectron

M125

FreeElectron

ocmcdizzle

M125

TheGovernment

FreeElectron

ocmcdizzle

wirk

M125

He1p1ess n00b

TheGovernment

wirk

Canis-X

FreeElectron

M125

Top Contributors this Month

Recommended Communities