Overclock.net banner
Cancel

[Build Log] Virtualized home server

14576 Views 54 Replies 14 Participants Last post by  Aximous, 
A
3
Hello OCN!
wave2.gif


This is the first buildlog I decided to start, tbh I was too lazy to take pictures and write stuff in the past, but hey I'm doing this rather than studying for the exam in the morning so here we go!

The reasoning:
My existing NAS filled up so I decided it was time to build a proper one, as this one was basically an HDD in the HTPC. Repurposing my old rig's hardware for this one, should be pretty reliable hopefully.

The hardware:
  • Rampage Extreme, first gen, socket 775, X48, pretty good board tbh, was kinda sad when I upgraded, kinda shame to run this in a server, I build with what I can and I can't afford a server board now
  • Q9450, should be plenty horsepower, plan is to run at stock clocks undervolted as much as possible
  • Corsair CX430v2
  • 8GB 1333 Kingston ECC UB ram, part number: KVR13E9K2/8I
  • Bunch of HDDs, what I have lying around and some new WD greens. Currently 1 of these: WD20EZRX, WD20EARX, Seagate 7200.11 500GB
  • Hyper 212+ for the CPU
Some of the above hardware is not yet here, but hopefully everything will be working (I'm little worried about the ram, the board is very picky with the ram). Also I'll update with the final hardware's part numbers.

Goals for the build:
As the title suggests the server will be virtualized, I'll be running ESXi on it with several guest os's, which will have to fulfill the following tasks:
  • pfSense as a router, as my old WRT54GL seems to be giving out slowly
    frown.gif
    also it seems much more robust than any other retail router, notable requirements are QoS and VPN
  • Ubuntu server:
    • PXE server as I'll be making the HTPC diskless
    • Media streaming and file sharing
    • Client backup
    • SVN and/or Git repos
    • SFTP/SSH access
    • MySQL and Postgres server
    • probably a LAMP stack, not sure yet on this one
    • Email server
    • Probably some monitoring too, not sure yet if I want it
  • unRaid for handling the storage needs and to provide some redundancy to it.
My plan is to get this thing up and running after I finish my exams by the beginning of February.

I'll try to provide the solutions here to all the problems I encounter, as I read up on some stuff I'm bound to encounter quite few of them, hopefully some of you will find this useful. Also I'll post a link to most the bash scripts and config files I'll end up writing.

The current state of the build:

Big one

Running it with a spare PSU and some ram from my rig as I'm waiting for those to arrive. I will be putting it my current case when I get a new one.

Index:
See less See more
Reply
Save
Like
1 - 20 of 55 Posts
N
you will need to make sure you set pfsense to boot straight away upon ESXI starting up and set ESXI to be static IP as it obviously wont get an IP until the pfsense system is booted
Reply
Save
Like
A
pfSense auto-starting is a nobrainer
smile.gif
Haven't really though about the IP of the box but it's will be either static IP or static DHCP, will test out which one works better.
See less See more
Reply
Save
Like
N
you will have to have static IP otherwise it wont be able to get an IP when it boots so you'll then not be able to manage it
Reply
Save
Like
A
I'll do that then, thanks for the heads up.
Reply
Save
Like
N
Are you going to be running any kind of raid to prevent a whole datastore being lost? the perc 5 and perc 6's are good, cheap and handle 8 SATA drives

If you want any help with ESXI just tell me, im still learning it but am fairly confident
smile.gif
See less See more
Reply
Save
Like
A
That's what unraid is there for, I'll be passing the hdd's via RDM to that VM and that will provide redundancy. I quite enjoy discovering what I can do with ESXi, I'll be sure to throw you a PM if I'm stuck, thanks
smile.gif
See less See more
Reply
Save
Like
N
Quote:
Originally Posted by Aximous View Post

That's what unraid is there for, I'll be passing the hdd's via RDM to that VM and that will provide redundancy. I quite enjoy discovering what I can do with ESXi, I'll be sure to throw you a PM if I'm stuck, thanks
smile.gif
Click to expand...
i am not sure how well that will work, effectively you are using unraid to make a network accessible datastore? so unraid system will have to boot before everything else (before pfsense) then boot the others up afterwards once the unraid datastores are available.

you are effectively going the below i think?

Physical drive > ESXI > Small datastore for Unraid OS > Multiple datastores just for unraid > ISCSI? > ESXI datastore > VM's
Reply
Save
Like
saranya21
or just pass the whole controller card through esxi and then assign it to the unraid vm which should then have access to all the drives (as if they were native) on the controller card. at least I think that's how it works.
Reply
Save
Like
A
I didn't invest in a controller card, will be using onboard. The VMs won't be on the unraid array so that won't be a problem, AFAIK it's not possible to keep the images on a location that only comes online with one of the VMs, I'll use a small drive to hold the VMs and the rest will be passed to the unraid VM.

The layout would be:
Small physical drive to hold VMs and act as datastores for all of them, pendrive to hold unraid as you can't bypass that because you need the GUID for the license, rest of the drives RDM'd to unraid and mounted as NFS shares everywhere else, which should be fine with correct boot order for the VMs. With RDM there won't be datastores for the drives, the whole point is that ESXi won't do anything with those drives other than passing it through to the VM as they are.
Reply
Save
Like
N
i suggest you at least use onboard for the VM datastore else if the drive fails you'll lose all the stuff
Reply
Save
Like
killabytes
Why use ECC RAM if your board doesn't support it?
Reply
Save
Like
A
Quote:
Originally Posted by Norse View Post

i suggest you at least use onboard for the VM datastore else if the drive fails you'll lose all the stuff
Click to expand...
I'm not sure what do you mean here. If you are referring to unraid on pendrive, I'll probably end up doing a workaround so it ends up as all the VMs on the drive, though the pendrive will still be required in the system. Actually it is pretty unlikely that it will kill the drive since it basically reads it when starting up and saves the settings when some setting changes and that's all the usage it does. If you are referring to all the VMs being stored on a single drive with no redundancy, I'll be running regular backups of them to the unraid array.
Quote:
Originally Posted by killabytes View Post

Why use ECC RAM if your board doesn't support it?
Click to expand...
The board does support ECC ram, only that they didn't update the memory support list. Hopefully it will work, if not I can return it anyway.
Reply
Save
Like
N
Quote:
Originally Posted by Aximous View Post

I'm not sure what do you mean here. If you are referring to unraid on pendrive, I'll probably end up doing a workaround so it ends up as all the VMs on the drive, though the pendrive will still be required in the system. Actually it is pretty unlikely that it will kill the drive since it basically reads it when starting up and saves the settings when some setting changes and that's all the usage it does. If you are referring to all the VMs being stored on a single drive with no redundancy, I'll be running regular backups of them to the unraid array.
The board does support ECC ram, only that they didn't update the memory support list. Hopefully it will work, if not I can return it anyway.
Click to expand...
Well ESXI needs to be able to store the files ie the virtual drives and such somewhere in the datastore which is where the OS and such will be installed

you basically go

Physical drive > a form of raid (not required but good idea) > ESXI datastore > VM setup using the datastore to store its virtual drive that the OS is then installed on

you seem to be going
physical drive > ESXI > unraid > making a fileshare? > OS's using the files?

Im not sure how you are going to be using unraid for anything other than some file storage?
See less See more
Reply
Save
Like
A
Yes the unraid array will only hold media, family photos, backups. The other VMs will work independently from that, only that they will be backed up the the array every now and then. I'll be doing the first scenario you wrote, without raid. Yes the unraid array will be shared and then the linux vm will stream content from that and back clients up the that array etc, but it won't store any of it's own stuff on there. I hope this clears it up.
Reply
Save
Like
E
This is exactly like something I was planning to build. ESXi with PfSense, FreeNAS or ZFSGuru off a flash drive with a 6 drive RAIDZ2 array for file storage (mostly media), a VM for torrenting, a VM for hosting a small webserver (mainly for me to learn), and another VM for testing and fooling around with. I was thinking of having all the Virtual machines installed on a SSD and have them backed-up to the ZFS array periodically and have the backups and my sensitive data (Less than 100 GB) backed up to an external hard drive for extra security. If for some reason the ZFS array and the external hard drive were to both fail (or got stolen or my house burned down), I'd have my extremely sensitive data encrypted and stored in the cloud (Less than 1 GB).

I can't afford this now, but I'm interested as to how this turns out. I've never used ESXi and I don't know much about Linux/BSD, but it should work. Keep us updated.
thumb.gif
See less See more
Reply
Save
Like
A
Thanks for your interested!

I was thinking about ZFS too but loosing the ability to easily expand the array turned me off, I can't afford to buy all the drives I'd need right now, and with unraid I can expand on the go. Your proposed setup looks pretty good, I'm not that worried about my storage dying and loosing the data, I had only one HDD developing bad sectors in the last 10(?) years and I could still get that off of that. Though you never can be too safe
smile.gif
See less See more
Reply
Save
Like
A
3
Update time!

The PSU, RAM, HDD and the pendrive arrived:

Big one

Installed the ram to see the compatibility:

Big one

It's running Prime95 right now, no problems so far, running at rated speed and timings, also testing how low I can go with the CPU it's currently at 1.19V at stock speeds, running pretty cool, 58°C max with only one fan running on low.

Once I'm done with final voltages and if it's 48 hour prime stable then I'll go back to setting up everything else, but it will probably have to wait some times as I have some pretty bad exams coming up
frown.gif


BTW sorry for the quality of the pics, forgot my DSLR at my parents' place.
See less See more
Reply
Save
Like
A
2
48 hour stress testing passed, everything is rock stable, temps are nice and cool, I'm pretty satisfied by the results, temps never exceeded 60°C even with 23°C room temps and it stayed pretty silent too.
wheee.gif


Next up will be ESXi setup, although I probably won't be able to work on this in the next 2 weeks because of my exams
gunner.gif
Expect frequent updates after that, I'll probably be working on this or I'll be pretty damn hungover.
See less See more
Reply
Save
Like
N
Quote:
Originally Posted by Aximous View Post

48 hour stress testing passed, everything is rock stable, temps are nice and cool, I'm pretty satisfied by the results, temps never exceeded 60°C even with 23°C room temps and it stayed pretty silent too.
wheee.gif


Next up will be ESXi setup, although I probably won't be able to work on this in the next 2 weeks because of my exams
gunner.gif
Expect frequent updates after that, I'll probably be working on this or I'll be pretty damn hungover.
Click to expand...
ESXI 5.1 i am assuming? though free version seems to max out at 8 cores per VM though when i "removed" the license key to test things it seemed to revert back to a 60 day trial
Reply
Save
Like
1 - 20 of 55 Posts
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Overclock.net
A forum community dedicated to overclocking enthusiasts and testing the limits of computing. Come join the discussion about computing, builds, collections, displays, models, styles, scales, specifications, reviews, accessories, classifieds, and more!
Full Forum Listing
Explore Our Forums
Hardware News Intel CPUs PC Gaming NVIDIA Water Cooling
Top