Overclock.net › Forums › Specialty Builds › Servers › Media/File Server Partitioning
New Posts  All Forums:Forum Nav:

Media/File Server Partitioning

post #1 of 12
Thread Starter 
Just added a software RAID array to my Linux dev box and I'm looking to gradually convert it to a dedicated media server. I went with software RAID because the hardware will definitely be changing once, if not several times, and I don't want to get tied to a specific hardware configuration/RAID controller.

Ubuntu 10.04
Asus P5K-E
Intel E2180 @ 3.3
2GB DDR2-1066
1 x 250GB boot drive
4 x 2TB software (mdadm) RAID 5

The array was created on the devices themselves /dev/sd[b-e], not on partitions.

I don't plan on having multiple partitions on the array, but I'm wondering if there's any technical/housekeeping advantage to putting the filesystem directly on /dev/md0 or to create a partition that spans the entire drive and put the filesystem on /dev/md0p1.

The filesystem will be XFS.
RAID0R
(14 items)
 
  
CPUMotherboardGraphicsRAM
i5 750 4.0GHz MSI P55-GD80 GTX 470 | 8800GT PhysX 2x2GB G.Skill Ripjaws 
Hard DriveOptical DriveCoolingOS
60GB Agility 2|1TB RAID0|1.5TB Pioneer DVR-217D XSPC Raystorm | XSPC RX240 Windows 7 Professional x64 
MonitorKeyboardPowerCase
27" Dell 2709W | 17" Samsung Logitech G15 Corsair HX850 Corsair 650D 
Mouse
Microsoft IntelliMouse 
  hide details  
Reply
RAID0R
(14 items)
 
  
CPUMotherboardGraphicsRAM
i5 750 4.0GHz MSI P55-GD80 GTX 470 | 8800GT PhysX 2x2GB G.Skill Ripjaws 
Hard DriveOptical DriveCoolingOS
60GB Agility 2|1TB RAID0|1.5TB Pioneer DVR-217D XSPC Raystorm | XSPC RX240 Windows 7 Professional x64 
MonitorKeyboardPowerCase
27" Dell 2709W | 17" Samsung Logitech G15 Corsair HX850 Corsair 650D 
Mouse
Microsoft IntelliMouse 
  hide details  
Reply
post #2 of 12
As far as I'm aware, you can't partition an MD RAID like that. I know hardware RAID controllers let you do it, but MD RAID doesn't.

What you can do however, is to use LVM, which will let you create logical volumes which are the same size or smaller than the size of your array. You add your array(s) to the LVM storage pool as Physical Volumes, then out of that pool create one or more Logical Volumes, which can be dynamically sized.

You then format the LVM volume(s), not the array(s) underneath.
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Apple Basic Keyboard Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Apple Basic Keyboard Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
post #3 of 12
Thread Starter 
In the 2.4 kernels you couldn't partition the array directly.

In 2.6 they added the ability, with the arrays taking the naming scheme:

/dev/md/dNN
/dev/md_dNN (I believe this is just a symlink)

Partition 1, on array 0 would be /dev/md/d0p1.

I ended up just throwing the filesystem directly on the array, tuned for my sunit and swidth.
RAID0R
(14 items)
 
  
CPUMotherboardGraphicsRAM
i5 750 4.0GHz MSI P55-GD80 GTX 470 | 8800GT PhysX 2x2GB G.Skill Ripjaws 
Hard DriveOptical DriveCoolingOS
60GB Agility 2|1TB RAID0|1.5TB Pioneer DVR-217D XSPC Raystorm | XSPC RX240 Windows 7 Professional x64 
MonitorKeyboardPowerCase
27" Dell 2709W | 17" Samsung Logitech G15 Corsair HX850 Corsair 650D 
Mouse
Microsoft IntelliMouse 
  hide details  
Reply
RAID0R
(14 items)
 
  
CPUMotherboardGraphicsRAM
i5 750 4.0GHz MSI P55-GD80 GTX 470 | 8800GT PhysX 2x2GB G.Skill Ripjaws 
Hard DriveOptical DriveCoolingOS
60GB Agility 2|1TB RAID0|1.5TB Pioneer DVR-217D XSPC Raystorm | XSPC RX240 Windows 7 Professional x64 
MonitorKeyboardPowerCase
27" Dell 2709W | 17" Samsung Logitech G15 Corsair HX850 Corsair 650D 
Mouse
Microsoft IntelliMouse 
  hide details  
Reply
post #4 of 12
heh, I didn't know they let you do that...I assume that it only works if you use partitionless array members like you do?


EDIT:

Just tried it on a pair of scratch drives. Let's you partition the array even if the array members have RAID autodetect partitions on them. Very nice.

Only caveat is that in Disk Utility when you partition the array it gives the message, "WARNING: The partition is misaligned by 244376 bytes. This may result in very poor performance. Repartitioning is suggested."

I don't understand why the partition would be misaligned, but I didn't notice any poor performance. The drives behaved the same as if they had no partitions on them.
Edited by parityboy - 8/21/11 at 11:54am
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Apple Basic Keyboard Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Apple Basic Keyboard Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
post #5 of 12
mkfs.xfs -f -i size=512,maxpct=0 /dev/md0
mkdir /storage
mount /dev/md0p1 /storage
mkdir /storage/{Media,File}

or

vi /etc/fstab/
/dev/md0p1 /storage xfs defaults,noatime 1 2


hth
Athena
(14 items)
 
Achilles
(9 items)
 
 
CPUMotherboardGraphicsHard Drive
[Intel] i5 2500k  [ASUS] p8p67 Evo B3 [EVGA] GTX 680 [Samsung] 1TB 
CoolingOSMonitorKeyboard
[Noctua] NH-D14 [Micro$oft] Win7 64-bit/Linux [HP] 2311x [Logitech] G510 
PowerCaseMouse
[XFX] 650W [Cooler Master] HAF X [Logitech] G400 
CPUMotherboardGraphicsRAM
990X 6.5Ghz EVGA X58 3 way SLI classified XFX GT 240/modded 8800GTS/8800GTX/5870/4980/FIR... Corsair Dominator PSC  
CoolingOSMonitorCase
F1 Extreme Dark, DICE and LN2!! XPeeeee and Win 7 Old CRT Open 
  hide details  
Reply
Athena
(14 items)
 
Achilles
(9 items)
 
 
CPUMotherboardGraphicsHard Drive
[Intel] i5 2500k  [ASUS] p8p67 Evo B3 [EVGA] GTX 680 [Samsung] 1TB 
CoolingOSMonitorKeyboard
[Noctua] NH-D14 [Micro$oft] Win7 64-bit/Linux [HP] 2311x [Logitech] G510 
PowerCaseMouse
[XFX] 650W [Cooler Master] HAF X [Logitech] G400 
CPUMotherboardGraphicsRAM
990X 6.5Ghz EVGA X58 3 way SLI classified XFX GT 240/modded 8800GTS/8800GTX/5870/4980/FIR... Corsair Dominator PSC  
CoolingOSMonitorCase
F1 Extreme Dark, DICE and LN2!! XPeeeee and Win 7 Old CRT Open 
  hide details  
Reply
post #6 of 12
Thread Starter 
Quote:
Originally Posted by parityboy View Post
Just tried it on a pair of scratch drives. Let's you partition the array even if the array members have RAID autodetect partitions on them. Very nice.
Note that autodetect partitions only work with metadata version 0.90. The newer versions break autodetect at the bootloader level, but store superblock info in areas that won't get blasted by an MBR/partition table. Sort of a trade off depending on your application.

Also sidenote, if you decide to use version 1.2, the --detail and --examine options return different values due to a bug in 2.6.7.1. (one returns 1.2, the other 01.02). This causes the automatic starting of the array to break, and forces a resync when you assemble it after fixing the error.

Quote:
Originally Posted by dhenzjhen View Post
mkfs.xfs -f -i size=512,maxpct=0 /dev/md0
mkdir /storage
mount /dev/md0p1 /storage
mkdir /storage/{Media,File}

hth
I didn't try that (already have everything setup), but if you're making the filesystem on /dev/md0, you won't want to mount /dev/md0p1

Also, with XFS on a RAID5 array, it's important to specify the sunit and swidth parameters based on your chunksize and number of drives. Some have reported 30-40% performance gain when created and mounted with the proper parameters. More info at https://raid.wiki.kernel.org/index.php/RAID_setup#XFS
RAID0R
(14 items)
 
  
CPUMotherboardGraphicsRAM
i5 750 4.0GHz MSI P55-GD80 GTX 470 | 8800GT PhysX 2x2GB G.Skill Ripjaws 
Hard DriveOptical DriveCoolingOS
60GB Agility 2|1TB RAID0|1.5TB Pioneer DVR-217D XSPC Raystorm | XSPC RX240 Windows 7 Professional x64 
MonitorKeyboardPowerCase
27" Dell 2709W | 17" Samsung Logitech G15 Corsair HX850 Corsair 650D 
Mouse
Microsoft IntelliMouse 
  hide details  
Reply
RAID0R
(14 items)
 
  
CPUMotherboardGraphicsRAM
i5 750 4.0GHz MSI P55-GD80 GTX 470 | 8800GT PhysX 2x2GB G.Skill Ripjaws 
Hard DriveOptical DriveCoolingOS
60GB Agility 2|1TB RAID0|1.5TB Pioneer DVR-217D XSPC Raystorm | XSPC RX240 Windows 7 Professional x64 
MonitorKeyboardPowerCase
27" Dell 2709W | 17" Samsung Logitech G15 Corsair HX850 Corsair 650D 
Mouse
Microsoft IntelliMouse 
  hide details  
Reply
post #7 of 12
Yeah I knew the newer metadata structure broke the autodetect, but didn't know why, so cheers for that. Metadata 1.2 has other advantages too, like being able to address up to 384 devices in a single array...not that you ever would, but bragging rights have their own inherent value.

Oddly enough, mdadm 2.6.7.1 is precisely the version I have (Ubuntu 10.04.2 LTS), but the while the --examine parameter works correctly with the array members, the --detail parameter results in
Code:
mdadm: /dev/md0 does not appear to be an md device
Maybe Canonical disabled the --detail code until Neil Brown fixed it? It can't be a metadata issue because I have arrays with both the 0.90 and 1.2 metadata variants.
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Apple Basic Keyboard Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Apple Basic Keyboard Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
post #8 of 12
Thread Starter 
Quote:
Originally Posted by parityboy View Post
Maybe Canonical disabled the --detail code until Neil Brown fixed it? It can't be a metadata issue because I have arrays with both the 0.90 and 1.2 metadata variants.
I WISH that was the case. I would have saved myself about 8 hours and 2 rebuilds.

The discrepancy shows up if you do a

mdadm --detail --scan
mdadm --examine --scan

Unfortunately, the numerous guides out there interchange those two commands when generating /etc/mdadm/mdadm.conf, so I found the bug the hard way.

I've come across one final "gotcha". When you use 1.2, the presence of the array name in mdadm.conf can also break autostarting. Per this thread (http://ubuntuforums.org/showthread.php?t=1764861) it seems like the only way to ensure proper autostart of the array is to use ONLY the ARRAY and UUID identifiers in the conf file. After that little name thing caused my 3rd rebuild in 2 days, I can finally reboot without issues.
RAID0R
(14 items)
 
  
CPUMotherboardGraphicsRAM
i5 750 4.0GHz MSI P55-GD80 GTX 470 | 8800GT PhysX 2x2GB G.Skill Ripjaws 
Hard DriveOptical DriveCoolingOS
60GB Agility 2|1TB RAID0|1.5TB Pioneer DVR-217D XSPC Raystorm | XSPC RX240 Windows 7 Professional x64 
MonitorKeyboardPowerCase
27" Dell 2709W | 17" Samsung Logitech G15 Corsair HX850 Corsair 650D 
Mouse
Microsoft IntelliMouse 
  hide details  
Reply
RAID0R
(14 items)
 
  
CPUMotherboardGraphicsRAM
i5 750 4.0GHz MSI P55-GD80 GTX 470 | 8800GT PhysX 2x2GB G.Skill Ripjaws 
Hard DriveOptical DriveCoolingOS
60GB Agility 2|1TB RAID0|1.5TB Pioneer DVR-217D XSPC Raystorm | XSPC RX240 Windows 7 Professional x64 
MonitorKeyboardPowerCase
27" Dell 2709W | 17" Samsung Logitech G15 Corsair HX850 Corsair 650D 
Mouse
Microsoft IntelliMouse 
  hide details  
Reply
post #9 of 12
Thread Starter 
Quote:
Originally Posted by C-bro View Post
... the hardware will definitely be changing once, if not several times, and I don't want to get tied to a specific hardware configuration/RAID controller.
Foresight truly is magnificent.

sudo mdadm --add /dev/md0 /dev/sde /dev/sdf
sudo mdadm --grow /dev/md0 --level=raid6 --raid-devices=6

Twelve hours later and a 6TB RAID5 array becomes an 8TB RAID6 array.

Don't buy into the hardware hoopla!
RAID0R
(14 items)
 
  
CPUMotherboardGraphicsRAM
i5 750 4.0GHz MSI P55-GD80 GTX 470 | 8800GT PhysX 2x2GB G.Skill Ripjaws 
Hard DriveOptical DriveCoolingOS
60GB Agility 2|1TB RAID0|1.5TB Pioneer DVR-217D XSPC Raystorm | XSPC RX240 Windows 7 Professional x64 
MonitorKeyboardPowerCase
27" Dell 2709W | 17" Samsung Logitech G15 Corsair HX850 Corsair 650D 
Mouse
Microsoft IntelliMouse 
  hide details  
Reply
RAID0R
(14 items)
 
  
CPUMotherboardGraphicsRAM
i5 750 4.0GHz MSI P55-GD80 GTX 470 | 8800GT PhysX 2x2GB G.Skill Ripjaws 
Hard DriveOptical DriveCoolingOS
60GB Agility 2|1TB RAID0|1.5TB Pioneer DVR-217D XSPC Raystorm | XSPC RX240 Windows 7 Professional x64 
MonitorKeyboardPowerCase
27" Dell 2709W | 17" Samsung Logitech G15 Corsair HX850 Corsair 650D 
Mouse
Microsoft IntelliMouse 
  hide details  
Reply
post #10 of 12
Which version of mdadm are you using?
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Apple Basic Keyboard Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
Ryzen
(12 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 7 1700 Gigabyte GA-AB350M Gaming 3 Palit GT-430 Corsair Vengeance LPX CMK16GX4M2B3000C15 
Hard DriveCoolingOSMonitor
Samsung 850 EVO AMD Wraith Spire Linux Mint 18.x Dell UltraSharp U2414H 
KeyboardPowerCaseMouse
Apple Basic Keyboard Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Servers
Overclock.net › Forums › Specialty Builds › Servers › Media/File Server Partitioning