Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › ZFS on linux. How stable?
New Posts  All Forums:Forum Nav:

ZFS on linux. How stable? - Page 2

post #11 of 20
Quote:
Originally Posted by Tadaen Sylvermane View Post

Its funny you mention Unix and the BSD's. I just started a new job and they rely on FreeBsd exclusively for all their servers. I was considering running FreeBsd myself both for the reliability, zfs, and the hands on experience I need to get anyway. The only thing that stops me is I was planning on doing an LTSP server for my media centers, and my MythTV backend. The LTSP I can live without but Myth I need. I haven't been able to virtualize it yet to see if it works because I have only tried on my laptop, MythTV is only in the ports and it takes forever to compile the damn thing when limited to a laptop, a virtual machine in a laptop no less.

I even looked into an NFS on root for my media centers but there is no package even in ports for the kodi-pvr-mythtv package so that is a bust.


*EDIT* Truth be told Mythtv is the only reason I was going to go with Ubuntu instead of Debian or CentOS.

Sir, if MythTV is all it takes to convert you to the one true path (= P) than I present you...
https://wiki.freebsd.org/MythTV
https://wiki.freebsd.org/HTPC
!!!

All pageantry aside, as far as I can tell MythTV runs quite well on FreeBSD. I wouldn't know, I run Kodi. However, freshports says we have it and had it for the past 10 years (port added in 2006). I am sure if it was broke as hell, we would flush it out of the repo. But there is a maintainer and I would give it a shot.
*Upon further reading, some things may not be there? Mainly the blasted tuner support. I would honestly give it trial run with your tuner and see where 2-3h of forum reads and mucking about get you? I mean imo its worth the time? If w/e you make works, post to forums! ^^

On the second part, you don't "need" to build it. Building from source is sweet, and I love doing it for some(*) things, but not everything needs to be build from source on FreeBSD. We have a package manager now, which went thru some revisions and works like a charm. I often use it for many things that I either suck at building (damn you Xorg! damn you to hell!) or have no need to build and am fine with default configurations.

If you got UNIX questions, I can try at least pointing you in a right place ^^ So never fear to ask!

@Petrol
You can compile for others yes, but it takes some degree of doing. You basically need to set up a local repo in the way the main freebsd servers are set up.
I've never done it (but definitely want to, because my machines range from top line AMD to stoneage laptops, and it would be nice to leverage FX-8350s horsepower to do this) but from what I can tell, people you this thing a lot.
http://www.freshports.org/ports-mgmt/poudriere


*Most of the time you need to build stuff when you want specific options enabled or disabled. For example, building OpenSSL is a good thing. Its really fat and has to large of an attack surface. Building it give you some degree of pruning it down.
Edited by _Red_Dog_ - 7/1/16 at 4:20pm
Phantom
(13 items)
 
Flagship
(11 items)
 
Vel'Koz
(11 items)
 
CPUMotherboardGraphicsRAM
Pentium 4 HP 3.0 GHz GA-8IG-1000-Pro-G SAPPHIRE ATI RadeOn 9600 Pro 128 MB Corsair VS512MB400 x 2 = 1024 Mb duel channel 
Hard DriveOptical DriveOSMonitor
Western Digital 120 GB SONY CD-RW CRX320E Windows XP MAG LCD 17' 
KeyboardPowerCaseMouse
Logisys InWin PowerMan 350W Phantom ( power suply was replaced ) Logitech Optical mouse 
CPUMotherboardGraphicsRAM
AMD FX-8350  Gigabyte GA‑990FXA‑UD5 Nvidia crucial ballistix tactical 
Hard DriveHard DriveOptical DriveCooling
Seagate Barracuda ST1000DM003 western digital red CD Combo Drive cooler master hyper 212 evo 
OSPowerCase
FreeBSD 10.2 COUGAR A-Series A560 Rosewill REDBONE Mid tower 
CPUMotherboardGraphicsRAM
AMD Athlon 5350 Kabini Quad-Core 2.05 GHz Biostar AM1MHP EVGA GeForce GT 730 LP Low Profile Graphics Car... 4gb corsair vengeance 1600mhz ddr3 
Hard DriveCoolingOSKeyboard
ADATA Premier SP550 64GB ARCTIC Alpine M1 FreeBSD 10.2 Iogear Multimedia GKM561R Wireless 2.4 GHz Keyb... 
PowerCaseMouse Pad
SeaSonic SS‑400ET DIYPC DIY-F2-P Integral Trackball 
  hide details  
Reply
Phantom
(13 items)
 
Flagship
(11 items)
 
Vel'Koz
(11 items)
 
CPUMotherboardGraphicsRAM
Pentium 4 HP 3.0 GHz GA-8IG-1000-Pro-G SAPPHIRE ATI RadeOn 9600 Pro 128 MB Corsair VS512MB400 x 2 = 1024 Mb duel channel 
Hard DriveOptical DriveOSMonitor
Western Digital 120 GB SONY CD-RW CRX320E Windows XP MAG LCD 17' 
KeyboardPowerCaseMouse
Logisys InWin PowerMan 350W Phantom ( power suply was replaced ) Logitech Optical mouse 
CPUMotherboardGraphicsRAM
AMD FX-8350  Gigabyte GA‑990FXA‑UD5 Nvidia crucial ballistix tactical 
Hard DriveHard DriveOptical DriveCooling
Seagate Barracuda ST1000DM003 western digital red CD Combo Drive cooler master hyper 212 evo 
OSPowerCase
FreeBSD 10.2 COUGAR A-Series A560 Rosewill REDBONE Mid tower 
CPUMotherboardGraphicsRAM
AMD Athlon 5350 Kabini Quad-Core 2.05 GHz Biostar AM1MHP EVGA GeForce GT 730 LP Low Profile Graphics Car... 4gb corsair vengeance 1600mhz ddr3 
Hard DriveCoolingOSKeyboard
ADATA Premier SP550 64GB ARCTIC Alpine M1 FreeBSD 10.2 Iogear Multimedia GKM561R Wireless 2.4 GHz Keyb... 
PowerCaseMouse Pad
SeaSonic SS‑400ET DIYPC DIY-F2-P Integral Trackball 
  hide details  
Reply
post #12 of 20
yeah I'm a Github user so I'm usually cloning repos and running the makefile, more often than not I end up having to grab dependencies from the package manager... and more often than not what I need is not in one of the repos on my list! Linux is annoying as hell but... well, things work. Things just work, one way or another. I really wanted to love BSD but the wireless drivers I need don't even support 802.11n let alone 802.11ac, which I'm mostly on these days.

That was an interesting article about the ZFS-ECC situation, maybe for my next NAS I'll go ZFS, ext4 doesn't really seem that necessary for a box hooked up to a UPS anyway
Black & Green
(12 items)
 
Dev Box
(7 items)
 
 
CPUMotherboardRAMHard Drive
Core2 Duo E7400 Asus P5Q Hyper-X  Sandisk 
OSPower
Fedora 22 Thermaltake 650W 
  hide details  
Reply
Black & Green
(12 items)
 
Dev Box
(7 items)
 
 
CPUMotherboardRAMHard Drive
Core2 Duo E7400 Asus P5Q Hyper-X  Sandisk 
OSPower
Fedora 22 Thermaltake 650W 
  hide details  
Reply
post #13 of 20
All I can say is I didn't really care for LVM. But I didn't spend too much time figuring it out, I took the easy way out.
My System
(21 items)
 
Server/HTPC
(11 items)
 
 
CPUMotherboardGraphicsRAM
AMD 8320 Asus m5a99fx pro EVGA 660ti  Gskill 8GB F3-1600C9-8GXM x2 
RAMHard DriveHard DriveHard Drive
4GB x2 OCZ Agility 3 Sasmsung 840 EVO Western Digital Caviar Blue 
Hard DriveOptical DriveOptical DriveCooling
Seagate 500gb Asus DRW-24B1ST Asus BC-12B1ST cool master hyper 212 evo 
OSOSMonitorMonitor
Windows 10 Pro x64 Arch  Asus 23" VH238 Asus 23" VH238H 
PowerCase
Corsair CX600M Fractal Design Define R5  
CPUMotherboardRAMHard Drive
i3 6100 Asus Z170M-Plus something 16gb DDR4 Western Digital 500GB 
Hard DriveHard DriveHard DriveOS
Samsung 2TB Western Digital Red 3TB HGST Deskter 4TB Unraid 6.x 
OSPowerCase
Ubuntu Server - VM Corsair CX430  Cooler Master HAF 912 
  hide details  
Reply
My System
(21 items)
 
Server/HTPC
(11 items)
 
 
CPUMotherboardGraphicsRAM
AMD 8320 Asus m5a99fx pro EVGA 660ti  Gskill 8GB F3-1600C9-8GXM x2 
RAMHard DriveHard DriveHard Drive
4GB x2 OCZ Agility 3 Sasmsung 840 EVO Western Digital Caviar Blue 
Hard DriveOptical DriveOptical DriveCooling
Seagate 500gb Asus DRW-24B1ST Asus BC-12B1ST cool master hyper 212 evo 
OSOSMonitorMonitor
Windows 10 Pro x64 Arch  Asus 23" VH238 Asus 23" VH238H 
PowerCase
Corsair CX600M Fractal Design Define R5  
CPUMotherboardRAMHard Drive
i3 6100 Asus Z170M-Plus something 16gb DDR4 Western Digital 500GB 
Hard DriveHard DriveHard DriveOS
Samsung 2TB Western Digital Red 3TB HGST Deskter 4TB Unraid 6.x 
OSPowerCase
Ubuntu Server - VM Corsair CX430  Cooler Master HAF 912 
  hide details  
Reply
post #14 of 20
Quote:
Originally Posted by Petrol View Post

yeah I'm a Github user so I'm usually cloning repos and running the makefile, more often than not I end up having to grab dependencies from the package manager... and more often than not what I need is not in one of the repos on my list! Linux is annoying as hell but... well, things work. Things just work, one way or another. I really wanted to love BSD but the wireless drivers I need don't even support 802.11n let alone 802.11ac, which I'm mostly on these days.

That was an interesting article about the ZFS-ECC situation, maybe for my next NAS I'll go ZFS, ext4 doesn't really seem that necessary for a box hooked up to a UPS anyway

The wireless support for BSDs and UNIX is something we always try to sweep under the rug and whistle innocently as we slowly walk away hehe.
You are right its pretty bad, however, honestly I got used to g over the years and roll with it nowdays. And yeah there is a ton of stuff that is just not supported period. After some 2h in front of openbsd manpages, google and ebay I kinda narrowed down some devices I am cool with, which is what I suggest everyone who wants use bsd on laptop to do. It sucks, but there are worst things. Odds are you will spent at most 15 bucks for said card and 1 to 3 hours of your time finding buying and installing the thing.

On the subject of building and difficulties... first, I'd start with just plain ports or pkg (or combination of those) and stick with it as opposed to using github. Freebsd has a github repo... but most of the progress/support goes on on their subversion repos. Funny we should bring this up, because on the latest bsdnow podcast someone brought up this thing
https://www.freshports.org/ports-mgmt/synth/
to use as a combination of pkg and port tools on single machine in a better, nicer fashion than just straight poudriere.

Also, if you just want a vault type of computer, try both dragonfly and freebsd. See if you like ZFS or Hammer better ^^. Most bsds install super quick and maybe an hour or so to look around and try some things, so maybe worth your time? *dragonfly also appears to have synth
Phantom
(13 items)
 
Flagship
(11 items)
 
Vel'Koz
(11 items)
 
CPUMotherboardGraphicsRAM
Pentium 4 HP 3.0 GHz GA-8IG-1000-Pro-G SAPPHIRE ATI RadeOn 9600 Pro 128 MB Corsair VS512MB400 x 2 = 1024 Mb duel channel 
Hard DriveOptical DriveOSMonitor
Western Digital 120 GB SONY CD-RW CRX320E Windows XP MAG LCD 17' 
KeyboardPowerCaseMouse
Logisys InWin PowerMan 350W Phantom ( power suply was replaced ) Logitech Optical mouse 
CPUMotherboardGraphicsRAM
AMD FX-8350  Gigabyte GA‑990FXA‑UD5 Nvidia crucial ballistix tactical 
Hard DriveHard DriveOptical DriveCooling
Seagate Barracuda ST1000DM003 western digital red CD Combo Drive cooler master hyper 212 evo 
OSPowerCase
FreeBSD 10.2 COUGAR A-Series A560 Rosewill REDBONE Mid tower 
CPUMotherboardGraphicsRAM
AMD Athlon 5350 Kabini Quad-Core 2.05 GHz Biostar AM1MHP EVGA GeForce GT 730 LP Low Profile Graphics Car... 4gb corsair vengeance 1600mhz ddr3 
Hard DriveCoolingOSKeyboard
ADATA Premier SP550 64GB ARCTIC Alpine M1 FreeBSD 10.2 Iogear Multimedia GKM561R Wireless 2.4 GHz Keyb... 
PowerCaseMouse Pad
SeaSonic SS‑400ET DIYPC DIY-F2-P Integral Trackball 
  hide details  
Reply
Phantom
(13 items)
 
Flagship
(11 items)
 
Vel'Koz
(11 items)
 
CPUMotherboardGraphicsRAM
Pentium 4 HP 3.0 GHz GA-8IG-1000-Pro-G SAPPHIRE ATI RadeOn 9600 Pro 128 MB Corsair VS512MB400 x 2 = 1024 Mb duel channel 
Hard DriveOptical DriveOSMonitor
Western Digital 120 GB SONY CD-RW CRX320E Windows XP MAG LCD 17' 
KeyboardPowerCaseMouse
Logisys InWin PowerMan 350W Phantom ( power suply was replaced ) Logitech Optical mouse 
CPUMotherboardGraphicsRAM
AMD FX-8350  Gigabyte GA‑990FXA‑UD5 Nvidia crucial ballistix tactical 
Hard DriveHard DriveOptical DriveCooling
Seagate Barracuda ST1000DM003 western digital red CD Combo Drive cooler master hyper 212 evo 
OSPowerCase
FreeBSD 10.2 COUGAR A-Series A560 Rosewill REDBONE Mid tower 
CPUMotherboardGraphicsRAM
AMD Athlon 5350 Kabini Quad-Core 2.05 GHz Biostar AM1MHP EVGA GeForce GT 730 LP Low Profile Graphics Car... 4gb corsair vengeance 1600mhz ddr3 
Hard DriveCoolingOSKeyboard
ADATA Premier SP550 64GB ARCTIC Alpine M1 FreeBSD 10.2 Iogear Multimedia GKM561R Wireless 2.4 GHz Keyb... 
PowerCaseMouse Pad
SeaSonic SS‑400ET DIYPC DIY-F2-P Integral Trackball 
  hide details  
Reply
post #15 of 20
I was just reading an article about the popularity of one particular 802.11g router. I remember having one myself but times change and now even 802.11ac is not enough bandwidth for me. The "theoretical max" is a bunch of bullplop frown.gif

Synth looks promising, anything curses-based can't be too bad thumb.gif I really want to give FreeBSD + ZFS a go on my NAS box but since I have no experience with either I'm apprehensive about trusting all my data to it and hoping I don't screw up some significant detail the first time around.
Black & Green
(12 items)
 
Dev Box
(7 items)
 
 
CPUMotherboardRAMHard Drive
Core2 Duo E7400 Asus P5Q Hyper-X  Sandisk 
OSPower
Fedora 22 Thermaltake 650W 
  hide details  
Reply
Black & Green
(12 items)
 
Dev Box
(7 items)
 
 
CPUMotherboardRAMHard Drive
Core2 Duo E7400 Asus P5Q Hyper-X  Sandisk 
OSPower
Fedora 22 Thermaltake 650W 
  hide details  
Reply
post #16 of 20
@Petrol

Its relatively difficult to eff something up all the way to data loss with ZFS.

First, don't chase speed at the start and just do raid-z 1 or 2.
Second set up smartd
Third set up scrub job on cron (we still have that as a separate daemon lol)
Fourth, don't shrink zpools. I think it can be done, but I wouldn't try it with live data.
Fifth, if your freebsd box falls down and you can not recover, just don't wipe the data. Boot with live usb, drop to shell, mount the ZFS and carry the data somewhere safe, than wipe.


For more real guru advise, ask others, I am still green ^^
P.S. Oh, and don't use "ZFS Evil Tuning guide". Not that it is wrong or without merit , no. Its just it has tons of warning such as "do at your own risk". As far as I understand, using it without having some experience with zfs is the shortest road to data loss.
Edited by _Red_Dog_ - 7/5/16 at 6:22am
Phantom
(13 items)
 
Flagship
(11 items)
 
Vel'Koz
(11 items)
 
CPUMotherboardGraphicsRAM
Pentium 4 HP 3.0 GHz GA-8IG-1000-Pro-G SAPPHIRE ATI RadeOn 9600 Pro 128 MB Corsair VS512MB400 x 2 = 1024 Mb duel channel 
Hard DriveOptical DriveOSMonitor
Western Digital 120 GB SONY CD-RW CRX320E Windows XP MAG LCD 17' 
KeyboardPowerCaseMouse
Logisys InWin PowerMan 350W Phantom ( power suply was replaced ) Logitech Optical mouse 
CPUMotherboardGraphicsRAM
AMD FX-8350  Gigabyte GA‑990FXA‑UD5 Nvidia crucial ballistix tactical 
Hard DriveHard DriveOptical DriveCooling
Seagate Barracuda ST1000DM003 western digital red CD Combo Drive cooler master hyper 212 evo 
OSPowerCase
FreeBSD 10.2 COUGAR A-Series A560 Rosewill REDBONE Mid tower 
CPUMotherboardGraphicsRAM
AMD Athlon 5350 Kabini Quad-Core 2.05 GHz Biostar AM1MHP EVGA GeForce GT 730 LP Low Profile Graphics Car... 4gb corsair vengeance 1600mhz ddr3 
Hard DriveCoolingOSKeyboard
ADATA Premier SP550 64GB ARCTIC Alpine M1 FreeBSD 10.2 Iogear Multimedia GKM561R Wireless 2.4 GHz Keyb... 
PowerCaseMouse Pad
SeaSonic SS‑400ET DIYPC DIY-F2-P Integral Trackball 
  hide details  
Reply
Phantom
(13 items)
 
Flagship
(11 items)
 
Vel'Koz
(11 items)
 
CPUMotherboardGraphicsRAM
Pentium 4 HP 3.0 GHz GA-8IG-1000-Pro-G SAPPHIRE ATI RadeOn 9600 Pro 128 MB Corsair VS512MB400 x 2 = 1024 Mb duel channel 
Hard DriveOptical DriveOSMonitor
Western Digital 120 GB SONY CD-RW CRX320E Windows XP MAG LCD 17' 
KeyboardPowerCaseMouse
Logisys InWin PowerMan 350W Phantom ( power suply was replaced ) Logitech Optical mouse 
CPUMotherboardGraphicsRAM
AMD FX-8350  Gigabyte GA‑990FXA‑UD5 Nvidia crucial ballistix tactical 
Hard DriveHard DriveOptical DriveCooling
Seagate Barracuda ST1000DM003 western digital red CD Combo Drive cooler master hyper 212 evo 
OSPowerCase
FreeBSD 10.2 COUGAR A-Series A560 Rosewill REDBONE Mid tower 
CPUMotherboardGraphicsRAM
AMD Athlon 5350 Kabini Quad-Core 2.05 GHz Biostar AM1MHP EVGA GeForce GT 730 LP Low Profile Graphics Car... 4gb corsair vengeance 1600mhz ddr3 
Hard DriveCoolingOSKeyboard
ADATA Premier SP550 64GB ARCTIC Alpine M1 FreeBSD 10.2 Iogear Multimedia GKM561R Wireless 2.4 GHz Keyb... 
PowerCaseMouse Pad
SeaSonic SS‑400ET DIYPC DIY-F2-P Integral Trackball 
  hide details  
Reply
post #17 of 20
Storage isn't really my forte but after skimming through a ZFS on Linux guide it seems pretty straightforward. It turns out OpenMediaVault (Debian) has a plugin for ZFS support so I guess that's my solution there. I really like OMV's web interface, ZFS was really the only thing missing. I guess it's time to start worrying about RAM :\
Black & Green
(12 items)
 
Dev Box
(7 items)
 
 
CPUMotherboardRAMHard Drive
Core2 Duo E7400 Asus P5Q Hyper-X  Sandisk 
OSPower
Fedora 22 Thermaltake 650W 
  hide details  
Reply
Black & Green
(12 items)
 
Dev Box
(7 items)
 
 
CPUMotherboardRAMHard Drive
Core2 Duo E7400 Asus P5Q Hyper-X  Sandisk 
OSPower
Fedora 22 Thermaltake 650W 
  hide details  
Reply
post #18 of 20
@Petrol

I'd use FreeNAS or FreeBSD, but whats right for you ^^

Just definitely than do not tune. Or trouble may arise... specially on w/e linux port of ZFS OMV is running now...
Phantom
(13 items)
 
Flagship
(11 items)
 
Vel'Koz
(11 items)
 
CPUMotherboardGraphicsRAM
Pentium 4 HP 3.0 GHz GA-8IG-1000-Pro-G SAPPHIRE ATI RadeOn 9600 Pro 128 MB Corsair VS512MB400 x 2 = 1024 Mb duel channel 
Hard DriveOptical DriveOSMonitor
Western Digital 120 GB SONY CD-RW CRX320E Windows XP MAG LCD 17' 
KeyboardPowerCaseMouse
Logisys InWin PowerMan 350W Phantom ( power suply was replaced ) Logitech Optical mouse 
CPUMotherboardGraphicsRAM
AMD FX-8350  Gigabyte GA‑990FXA‑UD5 Nvidia crucial ballistix tactical 
Hard DriveHard DriveOptical DriveCooling
Seagate Barracuda ST1000DM003 western digital red CD Combo Drive cooler master hyper 212 evo 
OSPowerCase
FreeBSD 10.2 COUGAR A-Series A560 Rosewill REDBONE Mid tower 
CPUMotherboardGraphicsRAM
AMD Athlon 5350 Kabini Quad-Core 2.05 GHz Biostar AM1MHP EVGA GeForce GT 730 LP Low Profile Graphics Car... 4gb corsair vengeance 1600mhz ddr3 
Hard DriveCoolingOSKeyboard
ADATA Premier SP550 64GB ARCTIC Alpine M1 FreeBSD 10.2 Iogear Multimedia GKM561R Wireless 2.4 GHz Keyb... 
PowerCaseMouse Pad
SeaSonic SS‑400ET DIYPC DIY-F2-P Integral Trackball 
  hide details  
Reply
Phantom
(13 items)
 
Flagship
(11 items)
 
Vel'Koz
(11 items)
 
CPUMotherboardGraphicsRAM
Pentium 4 HP 3.0 GHz GA-8IG-1000-Pro-G SAPPHIRE ATI RadeOn 9600 Pro 128 MB Corsair VS512MB400 x 2 = 1024 Mb duel channel 
Hard DriveOptical DriveOSMonitor
Western Digital 120 GB SONY CD-RW CRX320E Windows XP MAG LCD 17' 
KeyboardPowerCaseMouse
Logisys InWin PowerMan 350W Phantom ( power suply was replaced ) Logitech Optical mouse 
CPUMotherboardGraphicsRAM
AMD FX-8350  Gigabyte GA‑990FXA‑UD5 Nvidia crucial ballistix tactical 
Hard DriveHard DriveOptical DriveCooling
Seagate Barracuda ST1000DM003 western digital red CD Combo Drive cooler master hyper 212 evo 
OSPowerCase
FreeBSD 10.2 COUGAR A-Series A560 Rosewill REDBONE Mid tower 
CPUMotherboardGraphicsRAM
AMD Athlon 5350 Kabini Quad-Core 2.05 GHz Biostar AM1MHP EVGA GeForce GT 730 LP Low Profile Graphics Car... 4gb corsair vengeance 1600mhz ddr3 
Hard DriveCoolingOSKeyboard
ADATA Premier SP550 64GB ARCTIC Alpine M1 FreeBSD 10.2 Iogear Multimedia GKM561R Wireless 2.4 GHz Keyb... 
PowerCaseMouse Pad
SeaSonic SS‑400ET DIYPC DIY-F2-P Integral Trackball 
  hide details  
Reply
post #19 of 20
I got introduced to ZFS back when people still used Solaris, but never got too comfortable with it just due to lack of enough exposure. Years later, I heard about ZFS on FreeBSD and things like FreeNAS, but never used it. A few years ago, I had an opportunity to build a BackBlaze-like storage pod server and decided to use ZoL (ZFS on Linux) and it worked out really great on 45 drives with raidz2. But, i wasn't involved in day-to-day operations of that unit after deploying it. A couple of years ago, I ran into several debilitating issues with LVM2+RAID6 and decided to use ZFS again on Linux. I've now been using on a daily basis two separate arrays of 8x 3TB drives with ZFS in raidz2 and it has been pretty stable for the most part of 2 years.

I say "most" because there are some things to be aware of, but i can say that I have not lost any data yet unlike my situation with LVM2+RAID6.

First of all, ZFS thrives on using memory and if you don't set limits, it causes problems... ZFS/ARC is suppose to release memory when there's pressure to need more for running programs, but I found that it doesn't release fast enough and causes programs to die with out of memory errors. This is problematic in my case because I was running VMs with KVM. The workaround is to configure a limit to how much ram you will allow the ARC to use. On my 64GB server, I limit it to use no more than 16GB.

ZFS uses COW (Copy-On-Write), and so by it's nature, the more changes you make to a file, the more fragmented things get. This is the nature of the beast, and there's no such thing as defragmentation short of dumping all data and restoring it from scratch. For many use cases this doesn't pose a problem, but use cases like torrent downloading, where you're writing small chunks to a file all the time can become problematic over time. Over the long haul, you will see performance degradation as the fragmentation gets worse, and especially as the storage volume becomes fuller.

You have to practice fixing and responding to fault events; like simulate replacing a drive in a raidz volume, etc. It's not always smooth going.. for example, the 'zfs replace' command is really meant for you to replace failed "X" drive with "Y" drive, but if you replaced the physical drive, drive "Y" shows up in place of the failed drive "X", so you are in essence doing a replace of old "X" with new "X", and this really confuses zfs; the workaround involves pointing to drives by UUID or device path, etc. instead of /dev/sdX. Even then, it may require a "--force" option, so you have to get comfortable with all this and I recommend practicing it before you commit to using it. ZFS doesn't seem to "automatically" repair itself when a failed drive is replaced... at least not always.

The above are just from my own experiences, but i'm not suggesting you don't use ZFS... I use it because it has so many advantages that you simply don't get with other solutions. For one, I absolutely love the way snapshots are done in ZFS vs say LVM2... makes you almost think *** with LVM2 snapshots... I no longer have to think about how I want to pre-allocate my storage since it's all one big shared data pool and the ZFS data sets will simply adjust as the demand rises. Check out tools like zfs-auto-snapshot for taking rolling snapshots giving you a way to recover from situations where someone accidentally deleted an important file, or you want to compare a file's contents to last week's version of that file, etc. ZFS snapshots are so inexpensive... also, compression works great with certain data sets, i find that my virtual disk files compress really well and what use to take up over 300GB of space is now less than 150GB! I also share out certain ZFS data sets via NFS and the integration works great.
TAIPEI
(10 items)
 
AURORA
(13 items)
 
 
MotherboardGraphicsRAMHard Drive
ASRock X99 Extreme11 EVGA GTX 980 Superclocked 32GB 8x4GB Corsair LPX Samsung XP941  
Hard DriveCoolingOSMonitor
Western Digital 3TB RE Noctua NH-D15 Fedora 21 Linux Samsung S27D590C 
PowerCase
Seasonic SS-1200XP Cooler Master Cosmos II 
CPUMotherboardGraphicsRAM
Dual Quad-core L5430 2.66Ghz 12mb cache Intel 5000 chipset ATI ES1000 64GB FBDIMM DDR2 PC2-5300 667Mhz 
Hard DriveOSPower
WD3000FYYZ PERC H700 w/ 512MB cache CentOS 7.2.1511 950W x2 
  hide details  
Reply
TAIPEI
(10 items)
 
AURORA
(13 items)
 
 
MotherboardGraphicsRAMHard Drive
ASRock X99 Extreme11 EVGA GTX 980 Superclocked 32GB 8x4GB Corsair LPX Samsung XP941  
Hard DriveCoolingOSMonitor
Western Digital 3TB RE Noctua NH-D15 Fedora 21 Linux Samsung S27D590C 
PowerCase
Seasonic SS-1200XP Cooler Master Cosmos II 
CPUMotherboardGraphicsRAM
Dual Quad-core L5430 2.66Ghz 12mb cache Intel 5000 chipset ATI ES1000 64GB FBDIMM DDR2 PC2-5300 667Mhz 
Hard DriveOSPower
WD3000FYYZ PERC H700 w/ 512MB cache CentOS 7.2.1511 950W x2 
  hide details  
Reply
post #20 of 20
Lots of great stuff coming out in this thread, the more I read about ZFS though the deeper the rabbit hole seems to go :\
Black & Green
(12 items)
 
Dev Box
(7 items)
 
 
CPUMotherboardRAMHard Drive
Core2 Duo E7400 Asus P5Q Hyper-X  Sandisk 
OSPower
Fedora 22 Thermaltake 650W 
  hide details  
Reply
Black & Green
(12 items)
 
Dev Box
(7 items)
 
 
CPUMotherboardRAMHard Drive
Core2 Duo E7400 Asus P5Q Hyper-X  Sandisk 
OSPower
Fedora 22 Thermaltake 650W 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Linux, Unix
Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › ZFS on linux. How stable?