APOLLO (2CPU LGA1366 Server | InWin PP689 | 24 Disks Capacity) - by alpenwasser - Overclock.net - An Overclocking Community

Forum Jump: 

APOLLO (2CPU LGA1366 Server | InWin PP689 | 24 Disks Capacity) - by alpenwasser

Reply
 
Thread Tools
post #1 of 30 (permalink) Old 11-13-2013, 03:46 PM - Thread Starter
New to Overclock.net
 
alpenwasser's Avatar
 
Join Date: Jul 2013
Location: Switzerland
Posts: 557
Rep: 41 (Unique: 36)
aw--apollo--logo.png


Table of Contents


01. 2013-NOV-13: First Hardware Testing & The Noctua NH-U9DX 1366
02. 2013-NOV-16: Temporary Ghetto Setup, OS Installed
03. 2014-APR-01: PSU Mounting & LSI Controller Ghetto Test
04. 2014-APR-02: The Disk Racks
05. 2014-APR-08: Chipset Cooling & Adventures in Instability
06. 2014-APR-09: Disk Ventilation
07. 2014-APR-11: Fan Unit for Main Compartment Ventilation
08. 2014-APR-12: Storage Topology & Cabling
09. 2014 APR-26: Storage and Networking Performance
09. 2014-MAY-10: Sound Dampening & Final Pics


Wait, What, and Why?

So, yeah, another build. Another server, to be precise. Why? Well, as nice of a
system ZEUS is, it does have two major shortcomings for its use as a server.

When I originally conceived ZEUS, I did not plan on using ZFS (since it was not
yet production-ready on Linux at that point). The plan was to use ZEUS' HDDs as
single disks, backing up the important stuff. In case of a disk failure, the
loss of non-backed up data would have been acceptable, since it's mostly media
files. As long as there's an index of what was on the disk, that data could
easily be reaquired.

But right before ZEUS was done, I found out that ZFS was production-ready on
Linux, having kept a bit of an eye on it since fall 2012 when I dabbled in
FreeBSD and ZFS for the first time. Using FreeBSD on the server was not an
option though since I was nowhere near proficient enough with it to use it for
something that important, so it had to be Linux (that's why I didn't originally
plan on ZFS).

So, I deployed ZFS on ZEUS, and it's been working very nicely so far. However,
that brought with it two major drawbacks: Firstly, I was now missing 5 TB of
space, since I had been tempted by ZFS to use those for redundancy, even for our
media files. Secondly, and more importantly, ZEUS is not an ECC-memory-capable
system. The reason this might be a problem is that when ZFS verifies the data on
the disks, a corrupted bit in your RAM could cause a discrepancy between the
data in memory and the data on disk, in which case ZFS would "correct" the data
on your disk, therefore corrupting it. This is not exactly optimal IMO. How
severe the consequences of this would be in practice is an ongoing debate in
various ZFS threads I've read. Optimists estimate that it would merely corrupt
the file(s) with the concerned corrupt bit(s), pessimists are afraid it might
corrupt your entire pool.


The main focus of this machine will be:
  • room to install more disks over time
  • ECC-RAM capable
  • not ridiculously expensive
  • low-maintenance, high reliability and availability (within reason, it's still
    a home and small business server)


Hardware

The component choices as they stand now:
  • M/B: Supermicro X8DT3-LN4F
  • RAM: 12 GB ECC DDR3-1333 (Hynix)
  • CPUs: 2 x Intel L5630 Quad Cores, 40 W TDP each
  • Cooling: 2 x Noctua NH-UD9X 1366 (yes, air cooling! redface.gif )
  • Cooling: A few nice server double ball bearing San Ace fans will also
    be making an appearance.
  • Case: InWin PP689 (will be modded to fit more HDDs than in stock config)
  • Other: TBD


Modding

Instead of some uber-expensive W/C setup, the main part of actually building
this rig will be in modifying the PP689 for fitting as many HDDs as halfway
reasonable as neatly as possible. I have not yet decided if there will be
painting and/or sleeving and/or a window. A window is unlikely, the rest depends
mostly on how much time I'll have in the next few weeks (this is not a long-term
project, aim is to have it done way before HELIOS).

Also, since costs for this build should not spiral out of control, I will be
trying to reuse as many scrap and spare parts I have laying around as possible.


Teaser

More pics will follow as parts arrive and the build progresses, for now a shot of the
case:

(click image for full res)
aw--apollo--2013-11-07--01--pp689.jpeg


That's all for now, thanks for stopping by, and so long. smile.gif
alpenwasser is offline  
Sponsored Links
Advertisement
 
post #2 of 30 (permalink) Old 11-14-2013, 07:31 AM - Thread Starter
New to Overclock.net
 
alpenwasser's Avatar
 
Join Date: Jul 2013
Location: Switzerland
Posts: 557
Rep: 41 (Unique: 36)
First Steps


Hardware Tested

M/B, CPUs and memory have all arrived. The CPUs and M/B seem to be working OK.
One of the memory modules seems to be having a bit of trouble being recognized,
the other five work fine. I'll see if it's really defective or if it's just the
IT gods screwing with me a bit.


The Noctua NH9DX 1366

The Noctua NH-U9DX 1366 is a cooler from Noctua's series specifically made for
Xeon sockets. For those who don't know, LGA1366 sockets have an integrated
backplate, just like LGA2011, which makes them much more convenient than their
desktop counterparts. It's quite a nice and sturdy backplate, too, in fact it's
among the most solid backplates I've come across yet. This does, however,
require a slightly different mounting system. You just have four screws which
you bolt directly into the plate.

Aside from that, the cooler is identical to its desktop counterpart as far as I
know. Why the 92 mm version? For one thing, it was in stock, unlike the 120 mm
version of this cooler. Also, the CPUs only produce 40 W TDP each, so there
really is no need for high-end cooling. And as a bonus, I got supplied some
awesome San Ace fans with my case, which also happen to be 92 mm.

The Noctua fans which come with the cooler are just 3 pin fans (the newer models
of this cooler for LGA2011 come with a PWM fan I think), but the San Ace fans I
got with my case are actually PWM controlled! Since the M/B has a full set of
PWM headers (8, to be exact, how awesome is that!? biggrin.gif ) I will try the San Ace
fans and see how they play on lower rpm's (they run at 4,800 rpm on full speed
redface.gif ). This does not need to be a super-silent machine since it will be in its
own room, and since I really like the San Ace fans with regards to build quality
(and I'm a total sucker for build quality) I'd love to use them for this. The
Noctuas would admitteldy be better suited, but I'll see how things go with the
SA's first.


The Box

Unlike its shiny desktop counterparts, the NH-U9DX comes in a nice and subtle
(but sturdy) cardbord box with a simple sticker on it. I must admit I like this
box more than the shiny ones. smile.gif

(click image for full res)
aw--apollo--2013-11-14--01--noctua-box.jpeg


Contents

How it looks packaged...

(click image for full res)
aw--apollo--2013-11-14--02--noctua-contents.jpeg

... and out in the open.

(click image for full res)
aw--apollo--2013-11-14--03--noctua-contents.jpeg


Noctua Pr0n

A few glory shots of the cooler itself...

(click image for full res)
aw--apollo--2013-11-14--04--noctua-pr0n.jpeg

(click image for full res)
aw--apollo--2013-11-14--05--noctua-pr0n.jpeg


The San Ace 9G0912P1G09

There is no info about this fan on the web, I'm presuming it's something San Ace
makes specifically for InWin in an OEM deal.

I've hooked it up to a fan controller and got a max reading of 4,800 rpm, and
the Supermicro board turns them down to ~2,200 rpm on idle. They seem to be very
good fans, you can only really hear the sound of the air moving, no bearing or
motor noises so far. Also, they are heavy (~200 g per piece), which is always
nice for a build quality fetishist such as myself. biggrin.gif

Note: Hooking such a fan up to a desktop board as its power source would not be
advisable, they are rated for 1.1 A and might burn out the circuits on a desktop
board. Server boards usually have better fan power circuitry since they are
desinged with high-performance fans in mind. Just as a side note.

(click image for full res)
aw--apollo--2013-11-14--06--san-ace-92.jpeg


Compared to the Noctua fan which comes with the coolers. I might still go with
the Noctuas, but it's not the plan at the moment.

(click image for full res)
aw--apollo--2013-11-14--07--san-ace-noctua-92.jpeg


The Noctua NH-U9DX 1366 San Ace Edition

I had to improvise a bit with mounting the San Ace's to the tower. The clips
which you'd use with the Noctua fans rely on the fan having open corners, which
the San Ace's do not. Ah well, nothing a bit of cotton cord can't fix. biggrin.gif

(click image for full res)
aw--apollo--2013-11-14--08--noctua-san-ace-mounting.jpeg


And the current config in its full glory:

(click image for full res)
aw--apollo--2013-11-14--09--noctua-nh-u9dx-san-ace-edition.jpeg


Side note: The coolers were actually more expensive than the CPUs. :lol:


That's it for now, thanks for stopping by.
alpenwasser is offline  
post #3 of 30 (permalink) Old 11-15-2013, 04:11 PM
WHAT UP SOYBOIS
 
barkinos98's Avatar
 
Join Date: Mar 2012
Location: Champaign, IL
Posts: 4,763
Rep: 71 (Unique: 62)
Subbed for another build from you man...
Great work biggrin.gif

Pur Sang | Pur Esprit | Pur Cupio | Pur Laptop
Pur Sang
(25 items)
Pur Esprit
(11 items)
Macbook Pro 13"
(23 items)
CPU
Core i5-6600k
Motherboard
Z170i Pro Gaming
GPU
HD Graphics 530
RAM
XPG Z1 2x8GB 2800MHz
Hard Drive
OCZ RD400 128GB
Hard Drive
Samsung 840 Pro 256GB
Hard Drive
BarraCuda 2TB
Optical Drive
Samsung External
Power Supply
SuperNOVA 750G3
Cooling
Corsair H60 (2018)
Cooling
NZXT Grid
Cooling
Noctua NF-F12PWM Chromax
Case
Phanteks Enthoo Evolve ITX
Operating System
Windows 10 Education
Monitor
Asus PA248Q
Monitor
HP VH240a
Monitor
Vivo Dual Monitor Mount
Keyboard
Velocifire TKL78 & Granite Keyset
Keyboard
Logitech K400r
Mouse
Logitech MX Master
Mouse
Corsair Harpoon RGB
Mousepad
KupX Extended Edition
Audio
V-Moda Crossfade LP Phantom Chrome
Audio
Cyber Acoustics 2.1
Other
RGB Lighting
CPU
Pentium G4560
Motherboard
MSI H110i Pro
GPU
HD Graphics 510
RAM
Patriot Elite 1x4GB 2400MHz
Hard Drive
Silicon Power S55 60GB
Power Supply
EVGA 450BT
Cooling
Intel Stock Cooler
Case
SilverStone SG13B
Monitor
Samsung 42" 1080p
Keyboard
Logitech K400r
Other
TP-Link TL-WN881ND
CPU
Core i5-5257U
Motherboard
Logic Board
GPU
Iris Graphics 6100
RAM
8GB
Hard Drive
128GB
Hard Drive
My Passport 1TB
Hard Drive
easyStore 4TB
Power Supply
Magsafe 2 60W
Case
MacBook Pro Retina 13"
Operating System
macOS High Sierra
Monitor
Retina Display 13"
Keyboard
Backlit Keyboard
Mouse
Force Touch Trackpad
Audio
Anker SoundBuds Slim
Audio
JBL Flip 3
Audio
Bose QuietComfort QC25
Audio
Status Audio CB-1
Other
Thunderbolt Ethernet Adapter
Other
Apple Mini Displayport to VGA Adapter
Other
Griffin Mini Displayport to HDMI/DVI Adapter
Other
Society6 "Fallen Angel" Skin
Other
HP OfficeJet Pro 6978
Other
TP-Link Archer C50
▲ hide details ▲


barkinos98 is offline  
Sponsored Links
Advertisement
 
post #4 of 30 (permalink) Old 11-15-2013, 06:56 PM - Thread Starter
New to Overclock.net
 
alpenwasser's Avatar
 
Join Date: Jul 2013
Location: Switzerland
Posts: 557
Rep: 41 (Unique: 36)
Quote:
Originally Posted by barkinos98 View Post

Subbed for another build from you man...
Great work biggrin.gif

Thanks, I appreciate that! smile.gif

I got the RAM Thursday and the rig has passed several rounds of memtest by now, so I think
the hardware is OK. smile.gif
alpenwasser is offline  
post #5 of 30 (permalink) Old 11-16-2013, 10:31 AM - Thread Starter
New to Overclock.net
 
alpenwasser's Avatar
 
Join Date: Jul 2013
Location: Switzerland
Posts: 557
Rep: 41 (Unique: 36)
Up and Running, Ghetto Style


Hardware Validation

I've put the system together temporarily to validate the M/B, CPU and memory, so
far all seems good. A minimal Arch Linux setup has been installed and is
successfully running BOINC at the moment. smile.gif

EDIT:
I'm not running BOINC as a hardware validation tool, that's not what it's
designed to do. I have (mostly) validated the hardware and am now just running
BOINC.

Just to clarify. wink.gif
/EDIT

Gotta love low-power CPUs, core temps after about an hour of running BOINC on
all cores are:
31 C, 31 C, 35 C, 30 C,
32 C, 26 C, 29 C, 31 C


(click image for full res)
aw--apollo--2013-11-16--02--boinc.jpeg


Feast on the Ghetto-ness!

Yeah... biggrin.gif

(click image for full res)
aw--apollo--2013-11-16--01--ghetto-setup.jpeg


Next Up

I'll need to order some supplies for modding the front part of the case for more
HDDs. Still not sure if I'll paint it. Can't paint it in the apartment, and
temps in my workshop in the basement have dropped significantly since we now
have just a few degrees above freezing outside, so conditions for spray painting
are not optimal at all at the moment.
alpenwasser is offline  
post #6 of 30 (permalink) Old 11-27-2013, 10:50 PM
New to Overclock.net
 
Jakewat's Avatar
 
Join Date: Apr 2013
Location: New Zealand, Auckland
Posts: 450
Rep: 9 (Unique: 9)
Would you know any sites/ pages that I could find some good info on the whole server topic. I have a very faint idea of the whole thing but the specifics are something that I'm keen on studying. You seem to know your way around servers and wanted to ask where you acquired your knowledge.

Also I'll be having a look at this interesting build too.
Jakewat is offline  
post #7 of 30 (permalink) Old 11-28-2013, 02:25 AM - Thread Starter
New to Overclock.net
 
alpenwasser's Avatar
 
Join Date: Jul 2013
Location: Switzerland
Posts: 557
Rep: 41 (Unique: 36)
Quote:
Originally Posted by Jakewat View Post

Would you know any sites/ pages that I could find some good info on the whole server topic. I have a very faint idea of the whole thing but the specifics are something that I'm keen on studying. You seem to know your way around servers and wanted to ask where you acquired your knowledge.

Also I'll be having a look at this interesting build too.

Servethehome has some nice articles. They don't just look at the latest and greatest hardware, but also
stuff of interest to the home server enthusiast which you can get on eBay for cheaps (similar to the components
for this build). Also, just googling around for specific questions I might have.

Aside from that I have also found browsing the catalogues of server hardware vendors very helpful (Supermicro,
Tyan, Asus, LSI, Intel). It gives you a general idea of what components are available for what platform, and when
you find something that might be of interest to you, you can do more research on that specific part via Google and
see if you can get it on eBay (it can be kind of tricky to find the right stuff on eBay with generic searches such as
'lga 1366 server board' or something like that, lots of stuff gets missed which you only find with specific searches
such as 'Supermicro X8DT3' etc., you get the idea).

TBH though it's not all that complex, it's more about getting yourself acquainted with product lines which aren't
that known to the normal user (so, for example, finding out what kind of Xeons are available for the LGA1366
socket and which ones might be right for your needs), the procedure is pretty much the same as when a new
desktop product line is launched. The primary caveat is of course that info (reviews etc.) is more difficult to find,
sometimes even downright impossible, so you need to rely much more on spec sheets, manuals and the
occasional tidbit to find out whether or not some component (say, a M/B) is right for you. The nice part about
pro-grade hardware is that the documentation is often pretty good (for example, Supermicro's manual for my
motherboard has been very helpful), so you can often get a good idea of a product's capabilities by doing some
careful reading.

As for the software side of things (which could be argued to be the more important aspect): I've been using exclusively
Linux for quite a few years now (not to sound elitist, it's just a statement of fact wink.gif), and for most server-like tasks
there are some pretty decent resources available for Linux (I'm assuming for Win Server as well, but I'm not up-
to-date on that front), so usually I just have a look at the Arch Wiki, and if it's not in there I look around google
and Youtube to see if there are any tutorials available for what I wish to do (for instance, setting up a DHCP
server on my machine, although I'm not yet sure if I'll actually do that). On a side note: That's also pretty cool
about server-grade hardware: Good Linux support. smile.gif

Then there remains the topic of networking and security, for which I have found Eli the Computer Guy's channel
very helpful. Although I've only just started to delve deeper into that side of things, so there's still lots to learn.

For example, I recently did some research on Cisco since, for one thing, we have one of their routers, and secondly,
I was thinking about buying a managed switch from them. Then I found out that they'd done backdoor firmware
updates for a few of their router series and have been involved in quite a few controversies (for example, they
seem to have been helping China build its great firewall). There are also rumours that they've been helping
the NSA with their snooping around (say, sending info about their routers' users browsing and downloading
habits to the man?), although I must say that these are just rumours and I haven't been able to find anything
definite on that topic.

Still, I'm paranoid enough that this has motivated me to avoid prebuilt closed-source networking equipment
and now I want to implement my own solution to make sure my equipment only really does what I actually
tell it to do. However, I'm not yet far along in my research (and finances wink.gif) to implement something proper in
that regard. But yeah, I feel an urge to get rid of my Cisco router/access point and build my own equipment,
for which there remains a significant amount of work to do.


Sorry for the long post, but that's what I do, apparently rolleyes.gif

Cheers,
-aw
alpenwasser is offline  
post #8 of 30 (permalink) Old 11-28-2013, 02:36 AM
New to Overclock.net
 
Jakewat's Avatar
 
Join Date: Apr 2013
Location: New Zealand, Auckland
Posts: 450
Rep: 9 (Unique: 9)
The long posts are what separates you from most other forumers, which IMO is a good thing, not many people have the time and patience to do what you do and should be appreciated.
Anyway, thanks for the info. I will be sure to do a bit of research on the matter and as I have done to gain my knowledge of desktops, watch, read, and scroll through info smile.gif.
Jakewat is offline  
post #9 of 30 (permalink) Old 11-28-2013, 04:44 AM - Thread Starter
alpenwasser is offline  
post #10 of 30 (permalink) Old 11-30-2013, 05:54 AM
New to Overclock.net
 
DiGiCiDAL's Avatar
 
Join Date: Aug 2012
Location: Las Vegas
Posts: 2,682
Rep: 142 (Unique: 106)
On the more casual side of home servers you might want to check out WeGotServed.com - you won't find much detail on larger, more targeted server builds and apps on there (definitely more home-based than enterprise) but if you're looking for a solution for personal cloud, media streaming, backup repository, Intranet, groupware type servers it's a very noob-friendly site. More MS biased than many forums/blogs, but also more accessible than some of the more comprehensive ones - and definitely less of the "if you like a GUI you're a noob and not worth our time" type of thing you can run into on the enterprise-linux targeted sites sometimes. biggrin.gif

Not surprisingly, this is also looking like a thorough and unique build/project alpenwasser... subbed! thumb.gif
DiGiCiDAL is offline  
Reply

Quick Reply
Message:
Options

Register Now

In order to be able to post messages on the Overclock.net - An Overclocking Community forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.
User Name:
If you do not want to register, fill this field only and the name will be used as user name for your post.
Password
Please enter a password for your user account. Note that passwords are case-sensitive.
Password:
Confirm Password:
Email Address
Please enter a valid email address for yourself.
Email Address:

Log-in



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off