Overclock.net › Forums › Specialty Builds › Servers › ESXi storage choices
New Posts  All Forums:Forum Nav:

ESXi storage choices

post #1 of 14
Thread Starter 
I need help choosing storage for my ESXi server. I need fast storage that is fault tolerant which doesn't draw much power and isn't expensive, should be an easy request, right? tongue.gif Right now I have an IBM M1015 (no cache, no BBU) with drives in RAID 10, its meh.

Needs:
  • Use the storage solution for a long time (years)
  • Some kind of online community / support system
  • Fault-tolerant
  • Not draw huge amounts of power
  • Not exceed hardware/software budget of 600-700$ (arbitrary, can be changed but is my preference)
  • HDD and SSD arrays (maybe cache for HDDs too?)
  • How ever hardware is organized, it needs to be fast
  • Support for 10g and at least 1g connections if the storage will not live in the ESXi box
  • If a beefy RAID card or two is best then great, I have can dedicate 2 PCIe slots to storage. If not, I would prefer a rack-mountable solution



I'm open to any solution I suppose that is under $1K that will get me most if not all of what I want without the storage factored in (HDDs + SSDs). Recommendations?
Edited by Mr.N00bLaR - 9/12/17 at 8:05pm
5820K
(13 items)
 
  
CPUMotherboardGraphicsRAM
I7-5820k @ 4.5GHz ASUS X-99/USB 3.1 RX RS 480 8GB 1x8GB 2133MHz 
Hard DriveCoolingOSMonitor
Intel 535 120GB Noctua NH-D15 Windows 10 ASUS 24inch 1080p60 
KeyboardPowerCaseMouse
Microsoft natural 4000 Seasonic FP Series SSR-850FX Corsair Air 740 Logitech G700 
Mouse Pad
Ripper XXL 
  hide details  
Reply
5820K
(13 items)
 
  
CPUMotherboardGraphicsRAM
I7-5820k @ 4.5GHz ASUS X-99/USB 3.1 RX RS 480 8GB 1x8GB 2133MHz 
Hard DriveCoolingOSMonitor
Intel 535 120GB Noctua NH-D15 Windows 10 ASUS 24inch 1080p60 
KeyboardPowerCaseMouse
Microsoft natural 4000 Seasonic FP Series SSR-850FX Corsair Air 740 Logitech G700 
Mouse Pad
Ripper XXL 
  hide details  
Reply
post #2 of 14
subbed
Petitmort
(15 items)
 
  
CPUMotherboardGraphicsRAM
i9-7900x Asus Rampage VI Extreme EVGA GTX 1080 TI FTW3 iCX Hydro Copper G.SKILL TridentZ 32GB (4 x 8GB) DDR4 4133 
Hard DriveCoolingOSMonitor
Samsung 960 Pro NVMe M.2 1TB EK Water Loop Win 10 Dell 30" 
MonitorMonitorKeyboardPower
Atron 27" Gateway 24" Razer Blackwidow X Chroma Seasonic 1000w 
CaseMouseMouse Pad
Caselabs S8 Razer Naga Chroma I use 6 mousepads, lol 
  hide details  
Reply
Petitmort
(15 items)
 
  
CPUMotherboardGraphicsRAM
i9-7900x Asus Rampage VI Extreme EVGA GTX 1080 TI FTW3 iCX Hydro Copper G.SKILL TridentZ 32GB (4 x 8GB) DDR4 4133 
Hard DriveCoolingOSMonitor
Samsung 960 Pro NVMe M.2 1TB EK Water Loop Win 10 Dell 30" 
MonitorMonitorKeyboardPower
Atron 27" Gateway 24" Razer Blackwidow X Chroma Seasonic 1000w 
CaseMouseMouse Pad
Caselabs S8 Razer Naga Chroma I use 6 mousepads, lol 
  hide details  
Reply
post #3 of 14
How many disks and how much storage are we talking about here?

A decent LSI/Avago/Megaraid/3Ware/AMCC whatever they're calling themselves these days hardware RAID SAS controller with cache and BBU/capacitor and a "FastCache" license that automatically tiers SSD would give you some dynamite performance.

Stick it all in a Supermicro SAS rackmount server chassis, and you're pretty much set.


Another alternative would be to build out a FreeNAS or Windows 2012R2 storage server, skip the hardware RAID entirely, and present NFS/ISCSI targets to ESXi. The enterprise world is shifting away from hardware RAID in favor of disk pooling and logical storage management, e.g. ReFS in Windows or ZFS in Linux, etc.


I wrestled with this for years, and ended up going Hyper-V over SMB 3.0 because it was cheaper in the long run with Microsoft SPLA licensing.
post #4 of 14

Put a 10G nic in a synology and run SSD/HDD tiers and possibly SSD caching on the HDD tier ingest and egress. 

Gene-Z Dedi
(7 items)
 
Captain America
(6 items)
 
 
CPUMotherboardRAMHard Drive
i7 2600k @ 5.2 Ghz Asus Maximus IV Gene-Z Gskill 2133 4GB Caviar Black 
CoolingOSPower
Corsair H100 Arch x64 OCZ 650 Watt 
CPUCPUMotherboardGraphics
Intel Xeon E5-2650 Intel Xeon E5-2650 ASUS Z9PE-D8 WS XFX 6870 
RAMOS
32 GB Samsung Arch Nix 
  hide details  
Reply
Gene-Z Dedi
(7 items)
 
Captain America
(6 items)
 
 
CPUMotherboardRAMHard Drive
i7 2600k @ 5.2 Ghz Asus Maximus IV Gene-Z Gskill 2133 4GB Caviar Black 
CoolingOSPower
Corsair H100 Arch x64 OCZ 650 Watt 
CPUCPUMotherboardGraphics
Intel Xeon E5-2650 Intel Xeon E5-2650 ASUS Z9PE-D8 WS XFX 6870 
RAMOS
32 GB Samsung Arch Nix 
  hide details  
Reply
post #5 of 14
Thread Starter 
Quote:
Originally Posted by hammong View Post

How many disks and how much storage are we talking about here?

A decent LSI/Avago/Megaraid/3Ware/AMCC whatever they're calling themselves these days hardware RAID SAS controller with cache and BBU/capacitor and a "FastCache" license that automatically tiers SSD would give you some dynamite performance.

Stick it all in a Supermicro SAS rackmount server chassis, and you're pretty much set.


Another alternative would be to build out a FreeNAS or Windows 2012R2 storage server, skip the hardware RAID entirely, and present NFS/ISCSI targets to ESXi. The enterprise world is shifting away from hardware RAID in favor of disk pooling and logical storage management, e.g. ReFS in Windows or ZFS in Linux, etc.


I wrestled with this for years, and ended up going Hyper-V over SMB 3.0 because it was cheaper in the long run with Microsoft SPLA licensing.

I would say the solution would need support for many SATA devices. 8/10 to start, possible 14 or 16 to grow into, maybe more? RAID cards that offer great performance are great, not sure how much they retail for (or go used for).

I already have the server built actually and have been using it for two years (always with temporary storage solutions).

ESXi box:

Case: Rosewill RSV-L4500
Motherboard: SUPERMICRO MBD-X10DRI (should have gotten 10GB, oh well)
CPUs: 2x E5-2643v3 (3.2GHZ~3.4GHz) (eBay special thumb.gif)
RAM: Crucial 32GB (2 x 16GB) 288-Pin DDR4 SDRAM ECC DDR4 2133
Heatsinks: Noctuas
PSU: EVGa 850 P2 (not redundant but eh, I got a really good deal new)
iStar Rail kit


Drive pooling seems to be very efficient, possibly great if speed and redundancy is there. I did use FreeNAS briefly with iSCSI but I ended up removing that hardware from my rack and going in another direction. Maybe it deserves another look?
Edited by Mr.N00bLaR - 9/12/17 at 9:13pm
5820K
(13 items)
 
  
CPUMotherboardGraphicsRAM
I7-5820k @ 4.5GHz ASUS X-99/USB 3.1 RX RS 480 8GB 1x8GB 2133MHz 
Hard DriveCoolingOSMonitor
Intel 535 120GB Noctua NH-D15 Windows 10 ASUS 24inch 1080p60 
KeyboardPowerCaseMouse
Microsoft natural 4000 Seasonic FP Series SSR-850FX Corsair Air 740 Logitech G700 
Mouse Pad
Ripper XXL 
  hide details  
Reply
5820K
(13 items)
 
  
CPUMotherboardGraphicsRAM
I7-5820k @ 4.5GHz ASUS X-99/USB 3.1 RX RS 480 8GB 1x8GB 2133MHz 
Hard DriveCoolingOSMonitor
Intel 535 120GB Noctua NH-D15 Windows 10 ASUS 24inch 1080p60 
KeyboardPowerCaseMouse
Microsoft natural 4000 Seasonic FP Series SSR-850FX Corsair Air 740 Logitech G700 
Mouse Pad
Ripper XXL 
  hide details  
Reply
post #6 of 14
Thread Starter 
I've found some 9271's with BBU and cache + cachevault on ebay in the 150-200$ range. Seems like a faster card, thoughts?
5820K
(13 items)
 
  
CPUMotherboardGraphicsRAM
I7-5820k @ 4.5GHz ASUS X-99/USB 3.1 RX RS 480 8GB 1x8GB 2133MHz 
Hard DriveCoolingOSMonitor
Intel 535 120GB Noctua NH-D15 Windows 10 ASUS 24inch 1080p60 
KeyboardPowerCaseMouse
Microsoft natural 4000 Seasonic FP Series SSR-850FX Corsair Air 740 Logitech G700 
Mouse Pad
Ripper XXL 
  hide details  
Reply
5820K
(13 items)
 
  
CPUMotherboardGraphicsRAM
I7-5820k @ 4.5GHz ASUS X-99/USB 3.1 RX RS 480 8GB 1x8GB 2133MHz 
Hard DriveCoolingOSMonitor
Intel 535 120GB Noctua NH-D15 Windows 10 ASUS 24inch 1080p60 
KeyboardPowerCaseMouse
Microsoft natural 4000 Seasonic FP Series SSR-850FX Corsair Air 740 Logitech G700 
Mouse Pad
Ripper XXL 
  hide details  
Reply
post #7 of 14
Could you define what you mean by fault tolerant? Do you just mean supports some for or redundancy (RAID) or do you mean can act as shared storage for multiple ESXi hosts?
post #8 of 14
Thread Starter 
Quote:
Originally Posted by PuffinMyLye View Post

Could you define what you mean by fault tolerant? Do you just mean supports some for or redundancy (RAID) or do you mean can act as shared storage for multiple ESXi hosts?

I want to have the epectation that I can survive at least one drive failure but would prefer to surive two. This is not my back up plan, it is my stay up and runnig plan. Storage for multiple ESXi hosts would be pretty cool depsite only having one right now. I do plan on usong this as VM storage.
5820K
(13 items)
 
  
CPUMotherboardGraphicsRAM
I7-5820k @ 4.5GHz ASUS X-99/USB 3.1 RX RS 480 8GB 1x8GB 2133MHz 
Hard DriveCoolingOSMonitor
Intel 535 120GB Noctua NH-D15 Windows 10 ASUS 24inch 1080p60 
KeyboardPowerCaseMouse
Microsoft natural 4000 Seasonic FP Series SSR-850FX Corsair Air 740 Logitech G700 
Mouse Pad
Ripper XXL 
  hide details  
Reply
5820K
(13 items)
 
  
CPUMotherboardGraphicsRAM
I7-5820k @ 4.5GHz ASUS X-99/USB 3.1 RX RS 480 8GB 1x8GB 2133MHz 
Hard DriveCoolingOSMonitor
Intel 535 120GB Noctua NH-D15 Windows 10 ASUS 24inch 1080p60 
KeyboardPowerCaseMouse
Microsoft natural 4000 Seasonic FP Series SSR-850FX Corsair Air 740 Logitech G700 
Mouse Pad
Ripper XXL 
  hide details  
Reply
post #9 of 14
Quote:
Originally Posted by Mr.N00bLaR View Post

I want to have the epectation that I can survive at least one drive failure but would prefer to surive two. This is not my back up plan, it is my stay up and runnig plan. Storage for multiple ESXi hosts would be pretty cool depsite only having one right now. I do plan on usong this as VM storage.

And just to be clear this storage is going to reside on the same physical server that is your ESXi host correct? I'm just a little confused as to how you are intending to present this storage to the ESXi host on which the storage resides.
post #10 of 14
Thread Starter 
Quote:
Originally Posted by PuffinMyLye View Post

Quote:
Originally Posted by Mr.N00bLaR View Post

I want to have the epectation that I can survive at least one drive failure but would prefer to surive two. This is not my back up plan, it is my stay up and runnig plan. Storage for multiple ESXi hosts would be pretty cool depsite only having one right now. I do plan on usong this as VM storage.

And just to be clear this storage is going to reside on the same physical server that is your ESXi host correct? I'm just a little confused as to how you are intending to present this storage to the ESXi host on which the storage resides.

Not necessarily, I meant to leave it open - whatever the best solution is for what I need, another storage server or inside of the ESX. There benefits and drawbacks to both, not sure whats best for me. I hadnt considered a second ESXi host.
5820K
(13 items)
 
  
CPUMotherboardGraphicsRAM
I7-5820k @ 4.5GHz ASUS X-99/USB 3.1 RX RS 480 8GB 1x8GB 2133MHz 
Hard DriveCoolingOSMonitor
Intel 535 120GB Noctua NH-D15 Windows 10 ASUS 24inch 1080p60 
KeyboardPowerCaseMouse
Microsoft natural 4000 Seasonic FP Series SSR-850FX Corsair Air 740 Logitech G700 
Mouse Pad
Ripper XXL 
  hide details  
Reply
5820K
(13 items)
 
  
CPUMotherboardGraphicsRAM
I7-5820k @ 4.5GHz ASUS X-99/USB 3.1 RX RS 480 8GB 1x8GB 2133MHz 
Hard DriveCoolingOSMonitor
Intel 535 120GB Noctua NH-D15 Windows 10 ASUS 24inch 1080p60 
KeyboardPowerCaseMouse
Microsoft natural 4000 Seasonic FP Series SSR-850FX Corsair Air 740 Logitech G700 
Mouse Pad
Ripper XXL 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Servers
Overclock.net › Forums › Specialty Builds › Servers › ESXi storage choices