Originally Posted by TheBlackHole
Hi to everybody!
I encountered several problems with this mainboard and I hope someone here on the forum is able to help me. It will take some time to read all that I wrote but I think if you will be patient, you will agree that is definitely worth it!
I do have 2 ASUS systems Z9PE-D16 and Z9PE-D8 with 2x E5-2660 and 2xE5-2687W which i regularily change in between. Main memory is 256GB and 128 GB respectively, ECC 1600MHz.
To your questions:
1) The Q-Codes are listed in the manual and provide at least a solution to know until when the POST run ok. Good that the system is now starting
A good summary for initial boot issues:
2) If memory is "missing" then the first thing I would do is to reset the CMOS memory. Potentially, in trying to get the system starting, you changed essential settings and impacted this part of POST
3) ME is the Intel Management Engine which is part of Intels vPro offering. Basically it is a small computer on the motherboard supervising the operation of the system, even if the main CPUs are turned off - i.e. you can remotely start the Asus MB via this mechanism
4) With default CMOS settings, the MB shows the tarbet base frequency, which is in your case 3100MHz. If perf mode and similar things are changed, it could be different (haven't personally tried it)
5) This is normaly behavior. Even if the workload of AIDA, HWInfo etc is not big, you are executing software on your system. There is no hard affinity between the executing thread in an application abnd the physical core in the socket - shows up as a kind of random pattern. Benchmark software potentiall wakes up the CPU to get the timing operation right. I would *only* be concerned, if the system with good benchmark software under load would not go to either 3100MHz or better 3400 MHz (depending on your cooler). 3400 MHZ is the max frequency for all cores in Turbo mode. For 1-2 core under load the speed can go up to 3800 MHz. Good benchmark software for such a system is Intel's Linpack, be more critical of the results of the usual known benchmark apps - they are not optimized for NUMA boards like yours.
Memory amount: This is normal
Memory is used for all kind of things like I/O devices are mapped into physical memory space. Some utilities see one POV, some have other approaches. 48 MB difference look like a PCI express card is mapped into memory space.
4 x 8 GB memory is a suboptimal configuration for this system. One of the reasons for a dual socket MB is a the increased bandwidth you might get. In this configuration the 16 cores of the 2xE5-2687W will starve for memory bandwidth to achive their regular performance. Basically, you run the 8 core socket with the memory bandwidth of a 4 core i7-3770. Either switch to 8x4 GB or 8x8GB. Systems like this are NOT designed to achieve low latency /high speed for individual threads, for these kind of apps an OC system with fewer cores is the better solution. If you need THROUGHPUT, then this system will shine and and leave all single socket systems in the dust. As an example: It is possible to read a 100GB file within 5 seconds into main memory (this is actually what I am doing with mine). Try this with an OC i7-3770K or i7-3930K, it wouldn't be possible.
To identify possible errors, I'd start with one populated socket, and go with 1 RAM DIMM and start there. Run the system under load for a few hours (linpack i.e.) and check the stability in its minimal config. Move from there by adding more memory Dimms to one socket. If all goes well, THEN add the second. If problems arise, change back to a single socket config with the "faulty" CPU and re-check stability. If stability is there, your setup might have been screwed for dual socket ops (or one CPU is indeed defect. Clear the CMOS and re-check with 2 sockets.
As said, many classic benchmark utilities are "confused" with NUMA systems or if the performance goes in regions beyond the design of the software. Some of them get actual slower (as data has to be moved via the QPI interconnect and there are WAY too many cores to choose from. Software which does not explicitly control the mapping of threads and memory to the indivual sockets and cores, will never achieve top performance on a 32 core NUMA system. Most benchmark utilities are not written in this way, which regularily might people belief that the single socket system gives more bang for the buck. This is correct if the assessement is done with these kind of software, it is not correct with properly developed software.
This is especially true for memory bandwidth performance checks .Your decline might come from the utility's inability to properly understand your topology. Check with stream benchmark. A system with 1600MHz memory should be in the 80 GB/sec range
Q-Code 22: haven't checked with the Z9PE-D8, but the Z9PE-D16 displays 22 as normal system value in operation
Have you really checked that your power supply is delivering stable performance on the 12V CPU socket supplies? Did you change the connectors? The frequent uncoordinated reboots could lead into this direction.
Just to make sure I asked: Do you use retail versions of E5-2687W or engineering samples (ES)? If the latter, many of the issues could be from this choice.
AndyEdited by AndyE - 5/6/13 at 9:24am