Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Is there any way to make BTRFS more... resillient to power loss?
New Posts  All Forums:Forum Nav:

Is there any way to make BTRFS more... resillient to power loss?

post #1 of 7
Thread Starter 
Asking just what the title says. I recently started a thread about making a compressed file system image with dynamic space allocation and a set boundary size. I was able to accomplish what I sought with a sparse image formatted to BTRFS. This was working fine and dandy for a couple of weeks - but unfortunately power loss killed that (luckily I'm not horribly new at things - and this was just a testing image to make sure it could be used in a practical application.) The image will not mount. It complains about the tree blocks being corrupted... whether mounted directly or on a loop device.

Is there any sort of mount option I can use to alleviate this? I'm actually not certain it was power loss or if it was just a system lockup - as I hadn't touched the image in a while, but when I did go to mount it and bring the filesystem online - it wouldn't mount.

If I can't do anything about the reliability of BTRFS - are there other filesystems that one would suggest using? Keep in mind it has to support online filesystem compression.
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
post #2 of 7
In future you can mount your server onto a UPS. Failing that try ZFS.

No suggestions for fixing current problem though.
post #3 of 7
Thread Starter 
There's no chance of recovery and thats fine since it was just a testing image anyways. I have a UPS now, at the time I wasn't utilizing one but as I said - I'm uncertain of whether or not it was a power failure or if the system froze. I'll look into ZFS, hopefully it is a little more graceful. I've had (ex)fat(16,32), ext(2,3,4), and NTFS volumes survive multiple forced power cycles without failing - but BTRFS fails on the first one? Sounds like it truly isn't ready for prime time - yet distributions are switching to it? I find that hard to believe.
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
post #4 of 7
After trialling Btrfs on my laptop, I'm not convinced it isn't complete garbage.

ZFS on the other hand, I've had the same pool running for 5 or 6 years across a variety of different "servers" (ie a variety of different desktop PC mobos, CPUs, etc that were used as makeshift server parts) and OSs (Nexenta, OpenSolaris, FreeBSD). One of the servers even had a dodgy SATA controller that would randomly drop disks mid-write - it was so severe that it even caused the host OS to crash a few times. Yet I've never lost any data in ZFS.

I've become quite a fanboy with ZFS, but it's easier to set up than Btrfs, easier to maintain than Btrfs, has more features than any other file system on the market and has been the most resilient fs I've ever used (by a long way!).

The only draw back is it's requires a beefier machine than most file systems (64bit CPU and a few gigs of RAM are recommended) plus I personally would advise running it on either FreeBSD or an OpenIndiana kernel (if not OpenIndiana specifically). There are Linux ports of ZFS, but personally I wouldn't trust them for production systems as one is via FUSE, then other requires compiling your own kernel drivers to work around the GPL vs CDDL licence incompatibilities.
Edited by Plan9 - 8/13/13 at 12:47am
post #5 of 7
Thread Starter 
That's a problem for me - For one temperature is of concern with the machine in question (I'm working on this, but its a slow grueling process due to funding limitations) Secondly - Linux is required, so I can't rely on switching to a BSD (Known compatibility limitation of a commercial software package I need to run) Thirdly, memory is a problem on this machine I have 16gb of ram and consistantly find myself approaching that limitation - in fact its been the source of system lockups and I've been forced to implement a 16mb kernel level memory "blanket" so that it doesn't freeze. I'm going to switch to 32gb ECC memory to help alleviate this in the future.

ZFS sounds like it may actually be "more painful" to implement ZFS. I might have to just deal with constantly being in limbo between running out of hard drive space, running out of ram or overheating. Money needs to stop being my limitation. Unfortunately working 14-15 hour shifts on a contract with a fixed salary makes having a second job impossible. Whoever it was that decided the American military gets paid "too much" clearly has poor math skills.
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
post #6 of 7
Quote:
Originally Posted by Xaero252 View Post

That's a problem for me - For one temperature is of concern with the machine in question (I'm working on this, but its a slow grueling process due to funding limitations) Secondly - Linux is required, so I can't rely on switching to a BSD (Known compatibility limitation of a commercial software package I need to run) Thirdly, memory is a problem on this machine I have 16gb of ram and consistantly find myself approaching that limitation - in fact its been the source of system lockups and I've been forced to implement a 16mb kernel level memory "blanket" so that it doesn't freeze. I'm going to switch to 32gb ECC memory to help alleviate this in the future.

ZFS sounds like it may actually be "more painful" to implement ZFS. I might have to just deal with constantly being in limbo between running out of hard drive space, running out of ram or overheating. Money needs to stop being my limitation. Unfortunately working 14-15 hour shifts on a contract with a fixed salary makes having a second job impossible. Whoever it was that decided the American military gets paid "too much" clearly has poor math skills.

I run Linux VMs on FreeBSD - so you can get around the Linux-only requirement easily enough. But that would put additional stress on heat and memory usage - which is an issue already.

Given your requirements, I think ext4 would be your best option. Though I'd also recommend you look into XFS and SnapRAID. I don't have much experience with either of them two, but from what I've read, XFS is supposed to be good for low IO writes while also supporting some semi-advanced features like snapshotting. SnapRAID would work in place of something like LVM / mdadm and will allow you to grow your storage pool disk by disk with the option of redundancy - which might work well as a budge RAID for someone like yourself.

Just out of interest (nosiness), what's the commercial package that you run?
post #7 of 7
Thread Starter 
Unfortunately ext4 doesn't support transparent online compression. The actual volume is formated EXT4 because it works well and is rock solid; I have an encrypted image that I store confidential files in for customers etc... I'd like to keep that encrypted image (it's 1tb in size) compressed to help with its actual size on disk (it needs to be editable live so there goes SquashFS). Storing it in the btrfs container I made brought it down from 1tb on disk size to 300gb using LZO - quite a substantial space savings.

As far as the commercial package - It's a cryptography suite for encrypted connections to company servers; it relies on a pin-key combination system with a hardware dongle. Each dongle is associated with a single user and their pin and stores the certificates for the connection. Pretty big PITA to work with.
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Linux, Unix
Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Is there any way to make BTRFS more... resillient to power loss?