Originally Posted by Aximous
Originally Posted by KyadCK
Media is on the disk, not in RAM. What exactly do you think I'm writing 24/7 that the bits will sit in RAM long enough to do something? What proof do you have of corrupted info that actually continues to work?
Do you have MD5 proof, or do your RAID arrays just, die? Because frankly, if that were even remotely true, the data I have from servers over a decade old are either the biggest statistical anomaly in history, or my ISO of Brood War is just broken in ways I don't notice.
If ECC was a requirement
for either of those, Linux distros would not run them from bone stock configs. BTRS is OpenSUSE's standard "next next next" option. So that is 100% pure grain false info. And while we're on the topic, what makes ZFS and BTRS so prone to error as compared to, say, NTFS, that they would even try
to replace EXT with it? That logic makes no sense. There is nothing special about ZFS or BTRS.
ECC is good and helpful. Pretending it's a requirement is lying to yourself. "Questionable at best" is a massive massive
exaggeration, and if it were true, we wouldn't have non-ECC anymore.
No your data is not sitting in the ram, but the calculation about how to put your data on the disks (in case of software raid) has to use the system ram. If a flipped bit occurs there your data will be written to the disk incorrectly or become corrupted if you will.
Watch the video I link below and it's second part for MD5 proof, and detailed explanation of this topic. And yes, your Brood War ISO may be broken without you knowing it, it might appear completely fine, but if you check the MD5 sum or try to mount it might turn up corrupted.
NTFS is a completely different beast compared to ZFS and BTRFS, with NTFS you don't have to make any calculations about splitting and putting data back to gather to read and write from the disk since you are only handling a single disk with that. ZFS and BTRFS on the other hand has to calculate stuff like this because they span across multiple disks with striping. You could argue that ReiserFS does this too but, that addresses disks individually too, so this doesn't become a problem. Also this is an issue with standard raid controllers too, as those have to make the same calculations and use ram for these too, the only difference is that those have their own processor and ram to do these.
Also this is where the advantage of ZFS and alike come in, these file systems have built-in measures to detect and correct bit rot, which the HW raid controllers do not.
The videos I meantions:
Correct. Not for me though, because P410s.
Thank you for actual MD5 proof.
No it isn't. You can RAID0/STRIPE with NTFS
. Again, if this were such a big problem, we would not have moved on from EXT to BTRFS on OpenSUSE as the standard file system.
Standard RAID controllers with their own RAM use... Buffered ECC. In fact, the good ones even have battery backups (and they do complain if you disconnect them, loudly) and use Flash for their cache instead of volatile memory to ensure data loss due to power failure doesn't happen. Either way, see below.
So ZFS has error correction, making ECC less
of a requirement, not more. So I was right. The big issue with using normal RAM for FreeNAS to my understanding is because it will use as much RAM as it can to cache data, meaning it DOES stay in RAM at all times.
Originally Posted by wiretap
Just because someone hasn't experienced it, doesn't mean it doesn't exist. I've been a victim of bit-rot on my old Windows Home Server v1. I had several corrupt family photos, corrupt program installers, corrupt ISO images, and corrupt MP3 files. Luckily I had a backup on a set of dual layer DVD's at the time. You may not even realize you have bit-rot until months/years later when you go to open a file that you really need.. which is what happened to me. I now use a home server with ECC RAM + SnapRAID + Backblaze off-site backup. Since the components required for ECC support doesn't really cost much more than a normal desktop consumer grade system, I now build all my servers with ECC support. Bit-rot isn't some mythical creature, but it also doesn't happen all too often. When it does happen though, it really sucks and you'll probably wish you had spent a few extra dollars on a proper setup to prevent it. It's especially bad when it's in a RAID system and the parity gets calculated with bad data.. then the changes are irreversible.
You're right, it isn't some Mystical creature, but it also does not simply happen over time (and involve system RAM) either unless you're defragging your disk and actually writing data, which you should not do when archiving. If you are loosing data without writing anything, you have straight up data loss due to the storage medium; your HDD/SSD/Array is dying.
The funny thing to me is people arguing that this is something I wouldn't notice. You fail to understand my array.
My primary SSD array is run off a P410 with 1GB BBWB, and presented to the OS (ESXi) as one large drive (~1.2TB, it and the HDD arrays are both 4x 500GB in RAID5). All RAID functionality is provided by the controllers (3 of them). This disk is presented to ESXi, which claims the partitioning with VMFS. From there, I create VMs, which sit on top of the Hypervisor, and create their "disks" (files to ESX), which get presented to a VM. My main archiving server has two main "drives", both are located on different P410s. They are not set as software RAID in windows, the files are simply written to both; specifically to avoid software RAID failure, actually, and allow me to just reassign the VMDKs to another OS if I want to upgrade/change my storage server's OS.
If the RAM were to corrupt the data on the way to a P410 (which does it's RAID calculations without the System CPU/RAM, obviously), then the file wouldn't work upon initial write. I have two copies, plus my original. If one failed, it would be deleted and another copy made.
If the main "file" (the VMDK) were to be corrupted by a P410 against the MD5 file ESXi compares it to automatically, it would throw a warning. But they haven't. If the file in the NTFS/BTRFS/FS-of-choice inside the VMDK failed, the files wouldn't work, and in the case of the archive, I'd make a copy from the other RAID array.
In the event all that fails, Servers 1 and 2 take the occasional VMDK backups of one another, which is good, because if either server fails, I can assign the VMX to the server's roster and fire it up in seconds.
Originally Posted by Prophet4NO1
Originally Posted by Liranan
Found a lot of really cheap registered/buffered ECC RAM at less than 15 USD but need Opteron or Xeons for them. Non-registered/buffered RAM is more expensive but can still be found for around the same price as regular RAM and as I need more RAM for my server I think I'll get some anyway.
I can not speak for AMD, but on Intel some Celeron and Pentium chips support ECC. My freenas server was using a Pentium with ECC before the bigger Xeon was added for plex. My pfSense router is also using ECC with a $30 Celeron. You just need to check the Intel Ark page before you buy. Pretty sure there are a few i3 chips too.
AMD's chips can, but you need a motherboard that supports it. All 900-series Gigabyte boards do, as far as I am aware.
Correct, but be careful, not all of them do. Be certain the model you're buying.