Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Some questions about ZFS
New Posts  All Forums:Forum Nav:

Some questions about ZFS

post #1 of 8
Thread Starter 
Hi all, got some questions about ZFS filesystem on Ubuntu, some help would be great, thanks!


I used ZFS to create a storage pool on my Ubuntu system (12.04, 32bit) and has been using it as a file server for a few months now. Had a few hiccups - first, messed up Ubuntu so much that I had to do a reinstall, and then the OS drive just died last night, out of the blue, which has left my pool in a 'DEGRADED' state with some 'UNAVAILABLE' devices, as seen below:
Code:
pool: server
 state: DEGRADED
status: One or more devices could not be used because the label is missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
 scan: scrub in progress since Thu Nov  8 13:20:41 2012
    5.94T scanned out of 6.00T at 211M/s, 0h5m to go
    0 repaired, 98.97% done
config:

        NAME                   STATE     READ WRITE CKSUM
        server                 DEGRADED     0     0     0
          raidz1-0             ONLINE       0     0     0
            sda                ONLINE       0     0     0
            sdc                ONLINE       0     0     0
            sdd                ONLINE       0     0     0
            sde                ONLINE       0     0     0
        logs
          6973007206224077867  UNAVAIL      0     0     0  was /dev/sdb5
          sdb4                 ONLINE       0     0     0
        cache
          sdb7                 UNAVAIL      0     0     0
          sdb6                 UNAVAIL      0     0     0
          sdb3                 ONLINE       0     0     0

errors: No known data errors
According to some ZFS cheat sheets I've read, I should be able to remove cache devices from the pool but I just don't seem to be able to get rid of sdb7 and sdb6. Tried zpool replace and zpool remove, it said no such device in the pool when the output clearly says there is. My question is, is there any way to force remove these unavailable devices? Do I even have to remove them, considering there are other online devices in the same category? Is it bad to leave it in its degraded state?


Question 2, is it okay to add more drives controlled by a different chipset to this pool? My board only has 5 SATA ports and all are occupied right now. I can add a PCI/PCI E SATA card and thus more drives, but will this bring any compatibility issues with the existing pool? Would there be any huge performance issues?

Thanks in advance!
Chronos
(13 items)
 
Chaos
(8 items)
 
Eos
(10 items)
 
CPUMotherboardRAMHard Drive
AMD Llano A8 3850 2.9GHz MSI A75MA-P35 Socket FM1 mATX Kingston DDR3 8GB 1333MHz 1.5v WD Caviar Blue 500GB 
Hard DriveCoolingPowerCase
4x Samsung Spinpoint F4 2TB HD204UI Arctic Freezer 13 Corsair CX430 v2 X-Case RM 400/10- CS V3 4u 
CPUGraphicsGraphicsRAM
Core 2 Duo P8800 2.66 GHz GeForce 9400M GeForce 9600M GT 4GB DDR3 1066MHz 
Hard DriveOSMonitorCase
Corsair Force 3 SSD Mac OS X Mountain Lion 15" 1440x900 Aluminium Unibody 
  hide details  
Reply
Chronos
(13 items)
 
Chaos
(8 items)
 
Eos
(10 items)
 
CPUMotherboardRAMHard Drive
AMD Llano A8 3850 2.9GHz MSI A75MA-P35 Socket FM1 mATX Kingston DDR3 8GB 1333MHz 1.5v WD Caviar Blue 500GB 
Hard DriveCoolingPowerCase
4x Samsung Spinpoint F4 2TB HD204UI Arctic Freezer 13 Corsair CX430 v2 X-Case RM 400/10- CS V3 4u 
CPUGraphicsGraphicsRAM
Core 2 Duo P8800 2.66 GHz GeForce 9400M GeForce 9600M GT 4GB DDR3 1066MHz 
Hard DriveOSMonitorCase
Corsair Force 3 SSD Mac OS X Mountain Lion 15" 1440x900 Aluminium Unibody 
  hide details  
Reply
post #2 of 8
I think its still too buggy/beta code with ZFS on linux. If you like to go all in on zfs you should go with Solaris. The more you put in these storage pools, the more you risk.
Water-Rig
(23 items)
 
Chiller-Rig
(18 items)
 
 
CPUMotherboardGraphicsRAM
2600K Asus Maximus IV Extreme Gigabyte GTX 680 SO Corsair Dominator GT 2133 
Hard DriveHard DriveOSMonitor
OCZ Vertex 3 MaxIOPS  Corsair Force 2 Windows 7, x64 Ultimate BenQ 24" XL2410T (120hz) 
KeyboardPowerCaseMouse
SteelSeries 6GV2 Silverstone Strider Gold 750w CM Stacker 830 SteelSeries Xai Laser RUSE Gaming 
Mouse PadAudioOtherOther
SteelSeries Logitech G35 HWlabs Black-Ice SR-1 420 Radiator 3 x Prolimatech 140mm 
OtherOtherOtherOther
3 x Silverstone Airpenetrator 141 3x Silverstone 121FM Fans (gfx) 1x Swifttech MCP35x + MCP35x reservoir  EK Water Blocks Nichel-Acetal Supr. HF CPU 
OtherOther
EK Water Blocks Copper-Plexi Supr. HF VGA 2m ½" tube 
CPUMotherboardGraphicsRAM
P4 641 Asus Maximus Formula x38 Nvidia Geforce 7300LE Samsung PC5300 
Hard DriveCoolingCoolingCooling
Seagate 8x TECs, 50x50x41mm (288 couple) 35v 2x Phobya Temperature displays Neoprene tube 19/13 
CoolingCoolingCoolingCooling
4x Dominator GT waterblocks 1x EK CPU Supreme HF 2x EK DC 4 Pump  2x EK basic 250ml cylinder reservoirs 
CoolingCoolingCoolingOS
1x HWlabs GTX360 Scythe 3000 rpm 120x38mm 3x Enermax 120mm fans  Windows XP 
PowerCase
Enermax MaxRevo 1500W Fujitsu Siemens 
  hide details  
Reply
Water-Rig
(23 items)
 
Chiller-Rig
(18 items)
 
 
CPUMotherboardGraphicsRAM
2600K Asus Maximus IV Extreme Gigabyte GTX 680 SO Corsair Dominator GT 2133 
Hard DriveHard DriveOSMonitor
OCZ Vertex 3 MaxIOPS  Corsair Force 2 Windows 7, x64 Ultimate BenQ 24" XL2410T (120hz) 
KeyboardPowerCaseMouse
SteelSeries 6GV2 Silverstone Strider Gold 750w CM Stacker 830 SteelSeries Xai Laser RUSE Gaming 
Mouse PadAudioOtherOther
SteelSeries Logitech G35 HWlabs Black-Ice SR-1 420 Radiator 3 x Prolimatech 140mm 
OtherOtherOtherOther
3 x Silverstone Airpenetrator 141 3x Silverstone 121FM Fans (gfx) 1x Swifttech MCP35x + MCP35x reservoir  EK Water Blocks Nichel-Acetal Supr. HF CPU 
OtherOther
EK Water Blocks Copper-Plexi Supr. HF VGA 2m ½" tube 
CPUMotherboardGraphicsRAM
P4 641 Asus Maximus Formula x38 Nvidia Geforce 7300LE Samsung PC5300 
Hard DriveCoolingCoolingCooling
Seagate 8x TECs, 50x50x41mm (288 couple) 35v 2x Phobya Temperature displays Neoprene tube 19/13 
CoolingCoolingCoolingCooling
4x Dominator GT waterblocks 1x EK CPU Supreme HF 2x EK DC 4 Pump  2x EK basic 250ml cylinder reservoirs 
CoolingCoolingCoolingOS
1x HWlabs GTX360 Scythe 3000 rpm 120x38mm 3x Enermax 120mm fans  Windows XP 
PowerCase
Enermax MaxRevo 1500W Fujitsu Siemens 
  hide details  
Reply
post #3 of 8
Thread Starter 
Quote:
Originally Posted by qwwwizx View Post

I think its still too buggy/beta code with ZFS on linux. If you like to go all in on zfs you should go with Solaris. The more you put in these storage pools, the more you risk.
Thanks for the reply! I can't though. 1) The main purpose of this system is to be the media server for the house and I use Plex Media Server to do that. Turns out PMS isn't supported in Solaris so I'll have to store all data in a ZFS pool in Solaris, make it shared, set up a VM with Windows or another Linux distro with Plex support and run it through that. Too much hassle for me. 2) I've got A LOT of data in this pool already, I don't have a clue as to how I would migrate all of this over to a new OS (unless zpool export in Ubuntu and zpool import in Solaris works?) and 3) I'm not a Linux regular, still messing about and learning new stuff. I don't want to jump in to OS' I know nothing about. redface.gif
Chronos
(13 items)
 
Chaos
(8 items)
 
Eos
(10 items)
 
CPUMotherboardRAMHard Drive
AMD Llano A8 3850 2.9GHz MSI A75MA-P35 Socket FM1 mATX Kingston DDR3 8GB 1333MHz 1.5v WD Caviar Blue 500GB 
Hard DriveCoolingPowerCase
4x Samsung Spinpoint F4 2TB HD204UI Arctic Freezer 13 Corsair CX430 v2 X-Case RM 400/10- CS V3 4u 
CPUGraphicsGraphicsRAM
Core 2 Duo P8800 2.66 GHz GeForce 9400M GeForce 9600M GT 4GB DDR3 1066MHz 
Hard DriveOSMonitorCase
Corsair Force 3 SSD Mac OS X Mountain Lion 15" 1440x900 Aluminium Unibody 
  hide details  
Reply
Chronos
(13 items)
 
Chaos
(8 items)
 
Eos
(10 items)
 
CPUMotherboardRAMHard Drive
AMD Llano A8 3850 2.9GHz MSI A75MA-P35 Socket FM1 mATX Kingston DDR3 8GB 1333MHz 1.5v WD Caviar Blue 500GB 
Hard DriveCoolingPowerCase
4x Samsung Spinpoint F4 2TB HD204UI Arctic Freezer 13 Corsair CX430 v2 X-Case RM 400/10- CS V3 4u 
CPUGraphicsGraphicsRAM
Core 2 Duo P8800 2.66 GHz GeForce 9400M GeForce 9600M GT 4GB DDR3 1066MHz 
Hard DriveOSMonitorCase
Corsair Force 3 SSD Mac OS X Mountain Lion 15" 1440x900 Aluminium Unibody 
  hide details  
Reply
post #4 of 8
Quote:
Originally Posted by qwwwizx View Post

I think its still too buggy/beta code with ZFS on linux. If you like to go all in on zfs you should go with Solaris. The more you put in these storage pools, the more you risk.

or FreeBSD. or Debian GNU/kfreebsd
    
CPUMotherboardGraphicsRAM
AMD Phenom II X4 B55 Biostar A880GZ  AMD Radeon HD 4250 iGPU 8GB (2x4GB) Patriot Sector 5 DDR3 
Hard DriveHard DriveHard DriveOptical Drive
Seagate Barracuda 1TB (ST1000DM003) Western Digital Caviar SE 250GB Seagate External 1TB LG DVD Burner 
OSOSOSPower
Debian GNU/Linux 6.0.6 'Squeeze' 64-bit Arch Linux 64-bit FreeBSD 9.0 64-bit LPS Ultra 550 watt 
Case
Thermaltake V4  
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
AMD Phenom II X4 B55 Biostar A880GZ  AMD Radeon HD 4250 iGPU 8GB (2x4GB) Patriot Sector 5 DDR3 
Hard DriveHard DriveHard DriveOptical Drive
Seagate Barracuda 1TB (ST1000DM003) Western Digital Caviar SE 250GB Seagate External 1TB LG DVD Burner 
OSOSOSPower
Debian GNU/Linux 6.0.6 'Squeeze' 64-bit Arch Linux 64-bit FreeBSD 9.0 64-bit LPS Ultra 550 watt 
Case
Thermaltake V4  
  hide details  
Reply
post #5 of 8
Run FreeBSD or OpenIndiana with a virtualisation suite for Linux (even if that's just VirtualBox) for any apps you can't find native ports of.

Also, make sure you install a 64bit OS as ZFS is optimised for 64bit memory addressing. In fact 32bit CPUs aren't technically supported so that might be explain some of the buggy behaviour you're experiencing (that and running it on Ubuntu).
post #6 of 8
Thread Starter 
Quote:
Originally Posted by Plan9 View Post

Run FreeBSD or OpenIndiana with a virtualisation suite for Linux (even if that's just VirtualBox) for any apps you can't find native ports of.
Also, make sure you install a 64bit OS as ZFS is optimised for 64bit memory addressing. In fact 32bit CPUs aren't technically supported so that might be explain some of the buggy behaviour you're experiencing (that and running it on Ubuntu).
Nice to see you again, Plan9. thumb.gif

Unfortunately, can't switch to another OS just yet. But you're right about the x64 vs x86 thing. I had it running initially on 11.10 x64 and yesterday I was using 12.04 x86. There were a lot of glitches and errors besides this, so I gave up on it and went back to 11.10 x64, it's all running smoothly now.


Just to clear things up, I haven't had any issues with ZFS besides not being able to remove the unavailable cache and log devices from the devices list, which isn't really a big deal since they got replaced. It's the actual OS that usually gets messed up, by myself. redface.gif

Moving on, does someone have an answer to this:
Quote:
Question 2, is it okay to add more drives controlled by a different chipset to this pool? My board only has 5 SATA ports and all are occupied right now. I can add a PCI/PCI E SATA card and thus more drives, but will this bring any compatibility issues with the existing pool? Would there be any huge performance issues?
Chronos
(13 items)
 
Chaos
(8 items)
 
Eos
(10 items)
 
CPUMotherboardRAMHard Drive
AMD Llano A8 3850 2.9GHz MSI A75MA-P35 Socket FM1 mATX Kingston DDR3 8GB 1333MHz 1.5v WD Caviar Blue 500GB 
Hard DriveCoolingPowerCase
4x Samsung Spinpoint F4 2TB HD204UI Arctic Freezer 13 Corsair CX430 v2 X-Case RM 400/10- CS V3 4u 
CPUGraphicsGraphicsRAM
Core 2 Duo P8800 2.66 GHz GeForce 9400M GeForce 9600M GT 4GB DDR3 1066MHz 
Hard DriveOSMonitorCase
Corsair Force 3 SSD Mac OS X Mountain Lion 15" 1440x900 Aluminium Unibody 
  hide details  
Reply
Chronos
(13 items)
 
Chaos
(8 items)
 
Eos
(10 items)
 
CPUMotherboardRAMHard Drive
AMD Llano A8 3850 2.9GHz MSI A75MA-P35 Socket FM1 mATX Kingston DDR3 8GB 1333MHz 1.5v WD Caviar Blue 500GB 
Hard DriveCoolingPowerCase
4x Samsung Spinpoint F4 2TB HD204UI Arctic Freezer 13 Corsair CX430 v2 X-Case RM 400/10- CS V3 4u 
CPUGraphicsGraphicsRAM
Core 2 Duo P8800 2.66 GHz GeForce 9400M GeForce 9600M GT 4GB DDR3 1066MHz 
Hard DriveOSMonitorCase
Corsair Force 3 SSD Mac OS X Mountain Lion 15" 1440x900 Aluminium Unibody 
  hide details  
Reply
post #7 of 8
Quote:
Originally Posted by JCG View Post

Nice to see you again, Plan9. thumb.gif
cheers mate
Quote:
Originally Posted by JCG View Post

Question 2, is it okay to add more drives controlled by a different chipset to this pool? My board only has 5 SATA ports and all are occupied right now. I can add a PCI/PCI E SATA card and thus more drives, but will this bring any compatibility issues with the existing pool? Would there be any huge performance issues?
I've done just that myself and for the same reasons as you. My pool has been up and stable for ~9 months now
post #8 of 8
Thread Starter 
Quote:
Originally Posted by Plan9 View Post

cheers mate
I've done just that myself and for the same reasons as you. My pool has been up and stable for ~9 months now
Excellent! Just what I wanted to know. Thank you! thumb.gif
Chronos
(13 items)
 
Chaos
(8 items)
 
Eos
(10 items)
 
CPUMotherboardRAMHard Drive
AMD Llano A8 3850 2.9GHz MSI A75MA-P35 Socket FM1 mATX Kingston DDR3 8GB 1333MHz 1.5v WD Caviar Blue 500GB 
Hard DriveCoolingPowerCase
4x Samsung Spinpoint F4 2TB HD204UI Arctic Freezer 13 Corsair CX430 v2 X-Case RM 400/10- CS V3 4u 
CPUGraphicsGraphicsRAM
Core 2 Duo P8800 2.66 GHz GeForce 9400M GeForce 9600M GT 4GB DDR3 1066MHz 
Hard DriveOSMonitorCase
Corsair Force 3 SSD Mac OS X Mountain Lion 15" 1440x900 Aluminium Unibody 
  hide details  
Reply
Chronos
(13 items)
 
Chaos
(8 items)
 
Eos
(10 items)
 
CPUMotherboardRAMHard Drive
AMD Llano A8 3850 2.9GHz MSI A75MA-P35 Socket FM1 mATX Kingston DDR3 8GB 1333MHz 1.5v WD Caviar Blue 500GB 
Hard DriveCoolingPowerCase
4x Samsung Spinpoint F4 2TB HD204UI Arctic Freezer 13 Corsair CX430 v2 X-Case RM 400/10- CS V3 4u 
CPUGraphicsGraphicsRAM
Core 2 Duo P8800 2.66 GHz GeForce 9400M GeForce 9600M GT 4GB DDR3 1066MHz 
Hard DriveOSMonitorCase
Corsair Force 3 SSD Mac OS X Mountain Lion 15" 1440x900 Aluminium Unibody 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Linux, Unix
Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Some questions about ZFS