Overclock.net › Forums › Specialty Builds › Servers › For This Purpose: ECC Unnecessary?
New Posts  All Forums:Forum Nav:

For This Purpose: ECC Unnecessary?

post #1 of 10
Thread Starter 
My home server is currently a simple newsreader and file server, running a ZFS RAIDZ across a single vdev of 3x2TB plain old 7200.12 SATA drives.

I have recently started running a private Usenet indexer on it for myself and some friends. The indexer can chew up a decent amount of RAM on it's own, and at times will tap out my RAM and swap as well. Right now the server is running 4x1GB ECC modules, and I'm thinking I'm definitely going to want to upgrade to at least 8GB total if not 16GB.

At work I can get 8GB of non-ECC DDR3 for ~30. If I want to maintain ECC, it would cost ~65. Right about now I'm being extremely.....well, cheap. I'd like to spring for the non-ECC due to the cost, but not sure if I want to give up ECC given the ZFS array, which also stores the database for the indexer.


Thoughts?
    
CPUMotherboardGraphicsRAM
Core i7 970 @ 4.0 GHz 1.22 Vcore Asus Rampage II Gene GTX 260 216SP G.SKILL PI 3x2gb DDR3 1600 @ 7-8-7-24 
Hard DriveOSMonitorPower
2x 500gb Seagates RAID 0, 1x 500gb non-RAID Windows 7 Professional x64 ASUS 24'' VH242H / Spectre 24'' WS Corsair 750TX 
Case
Corsair 300R 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 970 @ 4.0 GHz 1.22 Vcore Asus Rampage II Gene GTX 260 216SP G.SKILL PI 3x2gb DDR3 1600 @ 7-8-7-24 
Hard DriveOSMonitorPower
2x 500gb Seagates RAID 0, 1x 500gb non-RAID Windows 7 Professional x64 ASUS 24'' VH242H / Spectre 24'' WS Corsair 750TX 
Case
Corsair 300R 
  hide details  
Reply
post #2 of 10
Stick with ECC if your using ZFS in my opinion.
post #3 of 10
@OP

Unless you consider your server to be mission critical, ECC isn't necessary. Yes, ZFS being what it is ECC would be better but hey, OSes cache things in RAM anyway so if non-ECC was that bad, we'd all be in trouble. smile.gif

Go for the 16GB. smile.gif

On a related note, I'm considering doing the same thing. You using newznab? How are you finding its abilities?
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Dell SK-8185 Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Dell SK-8185 Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
post #4 of 10
Quote:
Originally Posted by parityboy View Post


On a related note, I'm considering doing the same thing. You using newznab? How are you finding its abilities?

Just wanted to throw in that I have been using NN+ for about a week now. Running a Ubuntu Server 12.10 VM with 2 vCPU and 2GB Static RAM, on a Hyper-V host. I have gotten over 100k releases added so far, and things are really smooth. The Regex that are available with the plus version are so worth it, as very little would index without them. My index is currently my primary index site, and another private newznab index as a back up (limited to 250 api hits a day).
post #5 of 10

ECC Ram is great for server applications that are critical and MUST avoid downtime. However as we know ECC is a premium RAM. I think for what you are using your server for, ECC Ram is not a necessity and even with Regular RAM as long as you maintain the server the chance of downtime/memory errors is actually fairly slim. ECC is just an extra step better as IF something does go wrong it is correctable in memory and unlikely to result in downtime. 

 

Again for your purpose, it is not really needed.

post #6 of 10
Thread Starter 
Quote:
Originally Posted by parityboy View Post

@OP

Unless you consider your server to be mission critical, ECC isn't necessary. Yes, ZFS being what it is ECC would be better but hey, OSes cache things in RAM anyway so if non-ECC was that bad, we'd all be in trouble. smile.gif

Go for the 16GB. smile.gif

On a related note, I'm considering doing the same thing. You using newznab? How are you finding its abilities?

Yeah, my main concern here is ZFS, but you make a good point with normal caching ops.


And yep, I'm using NN+ - definitely worth the $20.

It's very well built, but I've noticed it is susceptible to issues backfilling if you throw it a large amount all at once. Baby steps and it's perfectly fine though. You'll want to make sure to run the optimise_db.php script fairly frequently, otherwise you'll be seeing 20GB+ sql databases.


I've almost completed 1400 days of backfill for most major newsgroups. Proper maintenance and my DB is only ~3GB or so. But it does chug a lot of RAM.
    
CPUMotherboardGraphicsRAM
Core i7 970 @ 4.0 GHz 1.22 Vcore Asus Rampage II Gene GTX 260 216SP G.SKILL PI 3x2gb DDR3 1600 @ 7-8-7-24 
Hard DriveOSMonitorPower
2x 500gb Seagates RAID 0, 1x 500gb non-RAID Windows 7 Professional x64 ASUS 24'' VH242H / Spectre 24'' WS Corsair 750TX 
Case
Corsair 300R 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 970 @ 4.0 GHz 1.22 Vcore Asus Rampage II Gene GTX 260 216SP G.SKILL PI 3x2gb DDR3 1600 @ 7-8-7-24 
Hard DriveOSMonitorPower
2x 500gb Seagates RAID 0, 1x 500gb non-RAID Windows 7 Professional x64 ASUS 24'' VH242H / Spectre 24'' WS Corsair 750TX 
Case
Corsair 300R 
  hide details  
Reply
post #7 of 10
@thread

I don't want to derail, but does newznab only pick up Usenet items where an NZB file already exists? I notice there are many posts on Usenet where an NZB file isn't included.
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Dell SK-8185 Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Dell SK-8185 Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
post #8 of 10
Quote:
Originally Posted by TurboTurtle View Post

Yeah, my main concern here is ZFS, but you make a good point with normal caching ops.


And yep, I'm using NN+ - definitely worth the $20.

It's very well built, but I've noticed it is susceptible to issues backfilling if you throw it a large amount all at once. Baby steps and it's perfectly fine though. You'll want to make sure to run the optimise_db.php script fairly frequently, otherwise you'll be seeing 20GB+ sql databases.


I've almost completed 1400 days of backfill for most major newsgroups. Proper maintenance and my DB is only ~3GB or so. But it does chug a lot of RAM.
Definitely true about backfilling. I had been getting the past 60 days with new usergroups, then I tried to backfill to the past 1500 days and it just didn't seem to work, even though I could see the script processing. Also, in case you aren't aware of the screen scripts that are included with NN+, the Newznab_Screen.sh script runs Update_Binaries, Update_Releases, and Optimise_db then will wait 10 minutes and run again. I just run that inside of a screen session and leave it be.

Quote:
Originally Posted by parityboy View Post

@thread

I don't want to derail, but does newznab only pick up Usenet items where an NZB file already exists? I notice there are many posts on Usenet where an NZB file isn't included.

I might be wrong, but I think the answer to your question is no. I believe NN actually creates the NZB during the Update_Releases script, from the binary headers it has gathered.
post #9 of 10
Thread Starter 
Quote:
Originally Posted by parityboy View Post

@thread

I don't want to derail, but does newznab only pick up Usenet items where an NZB file already exists? I notice there are many posts on Usenet where an NZB file isn't included.

NN will grab the headers from Usenet and determine if it is associated with any binaries. If it is, it will construct an NZB based on the headers. You can specify minimum post sizes to filter out posts with no files. Mine for example is 50MB. This filters 99% of spam and all of the empty posts.

If you want/need any other info feel free to PM me.
Quote:
Originally Posted by tycoonbob View Post

Definitely true about backfilling. I had been getting the past 60 days with new usergroups, then I tried to backfill to the past 1500 days and it just didn't seem to work, even though I could see the script processing. Also, in case you aren't aware of the screen scripts that are included with NN+, the Newznab_Screen.sh script runs Update_Binaries, Update_Releases, and Optimise_db then will wait 10 minutes and run again. I just run that inside of a screen session and leave it be.

I've modified the php scripts and run those in a screen session from a different script that just increments a few counters and loops those php scripts. Mostly automated. I imported the past 4 years up to June of last year, so I only have to backfill that far.
    
CPUMotherboardGraphicsRAM
Core i7 970 @ 4.0 GHz 1.22 Vcore Asus Rampage II Gene GTX 260 216SP G.SKILL PI 3x2gb DDR3 1600 @ 7-8-7-24 
Hard DriveOSMonitorPower
2x 500gb Seagates RAID 0, 1x 500gb non-RAID Windows 7 Professional x64 ASUS 24'' VH242H / Spectre 24'' WS Corsair 750TX 
Case
Corsair 300R 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 970 @ 4.0 GHz 1.22 Vcore Asus Rampage II Gene GTX 260 216SP G.SKILL PI 3x2gb DDR3 1600 @ 7-8-7-24 
Hard DriveOSMonitorPower
2x 500gb Seagates RAID 0, 1x 500gb non-RAID Windows 7 Professional x64 ASUS 24'' VH242H / Spectre 24'' WS Corsair 750TX 
Case
Corsair 300R 
  hide details  
Reply
post #10 of 10
@TurboTurtle

Cheers for that. It'll be a little while before I'm in a position to set up newznab, but when I do I'm sure I'll have questions for you. smile.gif
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Dell SK-8185 Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Dell SK-8185 Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Servers
Overclock.net › Forums › Specialty Builds › Servers › For This Purpose: ECC Unnecessary?