Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Help setting up a fake raid?
New Posts  All Forums:Forum Nav:

Help setting up a fake raid?

post #1 of 13
Thread Starter 
I could do with some help trying to setup a fake raid.

In my Linux box I've got a 2TB drive split into 1TB partitions and a 1TB drive.

The 2TB drive is my backup disk, one partition is for Time Machine and the other will be to backup the physical 1TB disk.

The problem I have now is that I need to get the physical 1TB backing up to the partitioned 1TB disk as if it was in RAID, or doing a backup once a day.

Does anyone know how I can go about doing this?

I'm using Ubuntu Natty and my drivers are partitioned as NTFS
Edited by Evostance - 10/2/11 at 1:55am
    
CPUMotherboardGraphicsRAM
Intel i7-4850HQ  Yes GT 750M 16GB Corsair 
Hard DriveHard DriveOSMonitor
512GB SSD 1TB Buffalo Drivestation OSX Mavericks 15" 2880 x 1800 IPS 
MouseOther
Magic Mouse Mobee Magic Charger 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Intel i7-4850HQ  Yes GT 750M 16GB Corsair 
Hard DriveHard DriveOSMonitor
512GB SSD 1TB Buffalo Drivestation OSX Mavericks 15" 2880 x 1800 IPS 
MouseOther
Magic Mouse Mobee Magic Charger 
  hide details  
Reply
post #2 of 13
are you wanting to setup an actual fakeraid or do you just want a script/cronjob that will run a daily backup?
Bazinga Punk
(12 items)
 
ooh shiny!
(6 items)
 
 
CPUMotherboardGraphicsRAM
Intel Xeon 3440 AsRock P55 extreme Evga 8800 GT 512 MB Gskill Ripjaws 
Hard DriveCoolingOSMonitor
Western Digital Blue Antec Khuler 620 Ubuntu 11.10 Asus vw264H 
KeyboardPowerCaseMouse
GIGABYTE KM7600 CORSAIR TX 650 Cooler Master 590 GIGABYTE GM-M6800 
CPUMotherboardGraphicsRAM
Intel Core I5 6500 Gigabyte z170xp-SLI Nvidia 970gtx Corsair 16gb ddr4 2666mhz  
Hard DriveOS
250gb Samsung Evo 850 Windows 10 & Ubuntu 15.10 
  hide details  
Reply
Bazinga Punk
(12 items)
 
ooh shiny!
(6 items)
 
 
CPUMotherboardGraphicsRAM
Intel Xeon 3440 AsRock P55 extreme Evga 8800 GT 512 MB Gskill Ripjaws 
Hard DriveCoolingOSMonitor
Western Digital Blue Antec Khuler 620 Ubuntu 11.10 Asus vw264H 
KeyboardPowerCaseMouse
GIGABYTE KM7600 CORSAIR TX 650 Cooler Master 590 GIGABYTE GM-M6800 
CPUMotherboardGraphicsRAM
Intel Core I5 6500 Gigabyte z170xp-SLI Nvidia 970gtx Corsair 16gb ddr4 2666mhz  
Hard DriveOS
250gb Samsung Evo 850 Windows 10 & Ubuntu 15.10 
  hide details  
Reply
post #3 of 13
Isnt ntfs under linux... slow?
post #4 of 13
Quote:
Originally Posted by Xinoxide View Post
Isnt ntfs under linux... slow?
ntfs-3g is fairly fast, if you are using it just for storage. anything of actual production value (running a server,database) its not ideal to use ntfs, in this case backing up, i wouldn't use a ntfs partition for it, i'd go at least ext3.
Bazinga Punk
(12 items)
 
ooh shiny!
(6 items)
 
 
CPUMotherboardGraphicsRAM
Intel Xeon 3440 AsRock P55 extreme Evga 8800 GT 512 MB Gskill Ripjaws 
Hard DriveCoolingOSMonitor
Western Digital Blue Antec Khuler 620 Ubuntu 11.10 Asus vw264H 
KeyboardPowerCaseMouse
GIGABYTE KM7600 CORSAIR TX 650 Cooler Master 590 GIGABYTE GM-M6800 
CPUMotherboardGraphicsRAM
Intel Core I5 6500 Gigabyte z170xp-SLI Nvidia 970gtx Corsair 16gb ddr4 2666mhz  
Hard DriveOS
250gb Samsung Evo 850 Windows 10 & Ubuntu 15.10 
  hide details  
Reply
Bazinga Punk
(12 items)
 
ooh shiny!
(6 items)
 
 
CPUMotherboardGraphicsRAM
Intel Xeon 3440 AsRock P55 extreme Evga 8800 GT 512 MB Gskill Ripjaws 
Hard DriveCoolingOSMonitor
Western Digital Blue Antec Khuler 620 Ubuntu 11.10 Asus vw264H 
KeyboardPowerCaseMouse
GIGABYTE KM7600 CORSAIR TX 650 Cooler Master 590 GIGABYTE GM-M6800 
CPUMotherboardGraphicsRAM
Intel Core I5 6500 Gigabyte z170xp-SLI Nvidia 970gtx Corsair 16gb ddr4 2666mhz  
Hard DriveOS
250gb Samsung Evo 850 Windows 10 & Ubuntu 15.10 
  hide details  
Reply
post #5 of 13
Thread Starter 
Quote:
Originally Posted by transhour View Post
are you wanting to setup an actual fakeraid or do you just want a script/cronjob that will run a daily backup?
Either, I don't mind.

NTFS seems pretty slow tbh.. (9mb/s across USB :/)

Formatting both paritions to ext4 as we speak. How would I fakeraid it then?
Edited by Evostance - 10/2/11 at 8:25am
    
CPUMotherboardGraphicsRAM
Intel i7-4850HQ  Yes GT 750M 16GB Corsair 
Hard DriveHard DriveOSMonitor
512GB SSD 1TB Buffalo Drivestation OSX Mavericks 15" 2880 x 1800 IPS 
MouseOther
Magic Mouse Mobee Magic Charger 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Intel i7-4850HQ  Yes GT 750M 16GB Corsair 
Hard DriveHard DriveOSMonitor
512GB SSD 1TB Buffalo Drivestation OSX Mavericks 15" 2880 x 1800 IPS 
MouseOther
Magic Mouse Mobee Magic Charger 
  hide details  
Reply
post #6 of 13
i think you want to setup a software raid, as fakeraid requires a motherboard bios that is capable of it, and you would be limiting the size of your drive to the smallest in the array (so your 2tb drive, would only be 1tb, and it will erase the drives clean.)

as for software raid, i've never set one up in linux, i've only done hardware and fake raid installations.

usb external drives have always been slow, i don't think you will see much of a boost in speed by going to ext3/4, and i think if you go software raid, it would drastically decrease your performance in linux.

there are several tools at your leisure that are available for linux tho, for backing up.

there is the formidable rsync, that you can setup in a cron job, that will at first make a mirror back up of your main drive, keeping permissions intact and file structure. after the first back up, it will then only back up files if changes are made to them and any new files (which will greatly increase the speed of the back ups.)

then there is tar, which is a compression program, that you can set to a cron job, that will "zip" your file system onto the external drive, there are several ways to do this, but it will back up the entire system, not just files that have changed since last back up.

there are several things you will need to take into account when you build any sort of cron job back up script, there are several folders in the linux filesystem you will not want to back up, as they are virtually populated and have drastically inflated space used (which more than likely stall any system if you try to back up these folders.), you will also want to avoid backing up the back up drive as well (it will enter into a loop trying to back it up, and more than likely crash or stall the system.)

i'm honestly not aware of any programs that are specifically designed to full back ups like this in linux, other than the two i've mentioned (which aren't really designed for this sole purpose, but will do the task that is needed.)

i will make a suggestion, that rsync is the better option here, and instead of backing up the entire installation, just backing up the /home folder might be the wiser course of action.

edit: here is a link on how to do a software raid for ubuntu 11.04

https://help.ubuntu.com/11.04/server...tallation.html

i do not vouch for the validity of this how to, other than it is on the ubuntu community site. i also take no responbility for misuse of the information contained in it or data loss that is suffered from it .
Edited by Transhour - 10/2/11 at 9:42am
Bazinga Punk
(12 items)
 
ooh shiny!
(6 items)
 
 
CPUMotherboardGraphicsRAM
Intel Xeon 3440 AsRock P55 extreme Evga 8800 GT 512 MB Gskill Ripjaws 
Hard DriveCoolingOSMonitor
Western Digital Blue Antec Khuler 620 Ubuntu 11.10 Asus vw264H 
KeyboardPowerCaseMouse
GIGABYTE KM7600 CORSAIR TX 650 Cooler Master 590 GIGABYTE GM-M6800 
CPUMotherboardGraphicsRAM
Intel Core I5 6500 Gigabyte z170xp-SLI Nvidia 970gtx Corsair 16gb ddr4 2666mhz  
Hard DriveOS
250gb Samsung Evo 850 Windows 10 & Ubuntu 15.10 
  hide details  
Reply
Bazinga Punk
(12 items)
 
ooh shiny!
(6 items)
 
 
CPUMotherboardGraphicsRAM
Intel Xeon 3440 AsRock P55 extreme Evga 8800 GT 512 MB Gskill Ripjaws 
Hard DriveCoolingOSMonitor
Western Digital Blue Antec Khuler 620 Ubuntu 11.10 Asus vw264H 
KeyboardPowerCaseMouse
GIGABYTE KM7600 CORSAIR TX 650 Cooler Master 590 GIGABYTE GM-M6800 
CPUMotherboardGraphicsRAM
Intel Core I5 6500 Gigabyte z170xp-SLI Nvidia 970gtx Corsair 16gb ddr4 2666mhz  
Hard DriveOS
250gb Samsung Evo 850 Windows 10 & Ubuntu 15.10 
  hide details  
Reply
post #7 of 13
Thread Starter 
Quote:
Originally Posted by transhour View Post
i think you want to setup a software raid, as fakeraid requires a motherboard bios that is capable of it, and you would be limiting the size of your drive to the smallest in the array (so your 2tb drive, would only be 1tb, and it will erase the drives clean.)

as for software raid, i've never set one up in linux, i've only done hardware and fake raid installations.

usb external drives have always been slow, i don't think you will see much of a boost in speed by going to ext3/4, and i think if you go software raid, it would drastically decrease your performance in linux.

there are several tools at your leisure that are available for linux tho, for backing up.

there is the formidable rsync, that you can setup in a cron job, that will at first make a mirror back up of your main drive, keeping permissions intact and file structure. after the first back up, it will then only back up files if changes are made to them and any new files (which will greatly increase the speed of the back ups.)

then there is tar, which is a compression program, that you can set to a cron job, that will "zip" your file system onto the external drive, there are several ways to do this, but it will back up the entire system, not just files that have changed since last back up.

there are several things you will need to take into account when you build any sort of cron job back up script, there are several folders in the linux filesystem you will not want to back up, as they are virtually populated and have drastically inflated space used (which more than likely stall any system if you try to back up these folders.), you will also want to avoid backing up the back up drive as well (it will enter into a loop trying to back it up, and more than likely crash or stall the system.)

i'm honestly not aware of any programs that are specifically designed to full back ups like this in linux, other than the two i've mentioned (which aren't really designed for this sole purpose, but will do the task that is needed.)

i will make a suggestion, that rsync is the better option here, and instead of backing up the entire installation, just backing up the /home folder might be the wiser course of action.

edit: here is a link on how to do a software raid for ubuntu 11.04

https://help.ubuntu.com/11.04/server...tallation.html

i do not vouch for the validity of this how to, other than it is on the ubuntu community site. i also take no responbility for misuse of the information contained in it or data loss that is suffered from it .
Thanks!

I think Rsync is for me. My Media drive im backing up doesnt store any of the OS so thats fine.

I guess if i just run
Code:
rsync --delete /media/1TB /media/BackUp
then that will keep it backed up the same.

Now I just need to get a cronjob done..
    
CPUMotherboardGraphicsRAM
Intel i7-4850HQ  Yes GT 750M 16GB Corsair 
Hard DriveHard DriveOSMonitor
512GB SSD 1TB Buffalo Drivestation OSX Mavericks 15" 2880 x 1800 IPS 
MouseOther
Magic Mouse Mobee Magic Charger 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Intel i7-4850HQ  Yes GT 750M 16GB Corsair 
Hard DriveHard DriveOSMonitor
512GB SSD 1TB Buffalo Drivestation OSX Mavericks 15" 2880 x 1800 IPS 
MouseOther
Magic Mouse Mobee Magic Charger 
  hide details  
Reply
post #8 of 13
Quote:
Originally Posted by Evostance View Post
Thanks!

I think Rsync is for me. My Media drive im backing up doesnt store any of the OS so thats fine.

I guess if i just run
Code:
rsync --delete /media/1TB /media/BackUp
then that will keep it backed up the same.

Now I just need to get a cronjob done..
might want to do something like:

rsync -ar --delete /media/1TB /media/Backup

as the -a switch, will turn it into archival mode (preserving timestamps, permissions, owner/groups,etc), -r switch makes it recursive.


first create a file called "backupscript.sh", and add the following:

Code:
#!/bin/bash
rsync -ar --delete /media/1TB /media/Backup
save the file, make it executable:

chmod +x ~/backupscript.sh

create a file, call it "backup.cron"

add the following to it:

Code:
1 2 3 4 5 /home/username/backupscript.sh
1- minute (0-59)
2- hour (0-23, 0 = midnight)
3- day (1-31)
4- month (1-12)
5- weekday (0-6, 0 = Sunday)

so if you want a script to run every day at midnight, it would be

Code:
0 0 * * *  /home/username/backupscript.sh
save the file, then run this command:

sudo crontab backup.cron
Edited by Transhour - 10/2/11 at 12:18pm
Bazinga Punk
(12 items)
 
ooh shiny!
(6 items)
 
 
CPUMotherboardGraphicsRAM
Intel Xeon 3440 AsRock P55 extreme Evga 8800 GT 512 MB Gskill Ripjaws 
Hard DriveCoolingOSMonitor
Western Digital Blue Antec Khuler 620 Ubuntu 11.10 Asus vw264H 
KeyboardPowerCaseMouse
GIGABYTE KM7600 CORSAIR TX 650 Cooler Master 590 GIGABYTE GM-M6800 
CPUMotherboardGraphicsRAM
Intel Core I5 6500 Gigabyte z170xp-SLI Nvidia 970gtx Corsair 16gb ddr4 2666mhz  
Hard DriveOS
250gb Samsung Evo 850 Windows 10 & Ubuntu 15.10 
  hide details  
Reply
Bazinga Punk
(12 items)
 
ooh shiny!
(6 items)
 
 
CPUMotherboardGraphicsRAM
Intel Xeon 3440 AsRock P55 extreme Evga 8800 GT 512 MB Gskill Ripjaws 
Hard DriveCoolingOSMonitor
Western Digital Blue Antec Khuler 620 Ubuntu 11.10 Asus vw264H 
KeyboardPowerCaseMouse
GIGABYTE KM7600 CORSAIR TX 650 Cooler Master 590 GIGABYTE GM-M6800 
CPUMotherboardGraphicsRAM
Intel Core I5 6500 Gigabyte z170xp-SLI Nvidia 970gtx Corsair 16gb ddr4 2666mhz  
Hard DriveOS
250gb Samsung Evo 850 Windows 10 & Ubuntu 15.10 
  hide details  
Reply
post #9 of 13
Thread Starter 
Thanks for that

I've set it to run at 3am so fingers crossed it'll be ok
    
CPUMotherboardGraphicsRAM
Intel i7-4850HQ  Yes GT 750M 16GB Corsair 
Hard DriveHard DriveOSMonitor
512GB SSD 1TB Buffalo Drivestation OSX Mavericks 15" 2880 x 1800 IPS 
MouseOther
Magic Mouse Mobee Magic Charger 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Intel i7-4850HQ  Yes GT 750M 16GB Corsair 
Hard DriveHard DriveOSMonitor
512GB SSD 1TB Buffalo Drivestation OSX Mavericks 15" 2880 x 1800 IPS 
MouseOther
Magic Mouse Mobee Magic Charger 
  hide details  
Reply
post #10 of 13
Quote:
Originally Posted by transhour View Post
might want to do something like:

rsync -ar --delete /media/1TB /media/Backup

as the -a switch, will turn it into archival mode (preserving timestamps, permissions, owner/groups,etc), -r switch makes it recursive.


first create a file called "backupscript.sh", and add the following:

Code:
#!/bin/bash
rsync -ar --delete /media/1TB /media/Backup
save the file, make it executable:

chmod +x ~/backupscript.sh

create a file, call it "backup.cron"

add the following to it:

Code:
1 2 3 4 5 /home/username/backupscript.sh
1- minute (0-59)
2- hour (0-23, 0 = midnight)
3- day (1-31)
4- month (1-12)
5- weekday (0-6, 0 = Sunday)

so if you want a script to run every day at midnight, it would be

Code:
0 0 * * *  /home/username/backupscript.sh
save the file, then run this command:

sudo crontab backup.cron
This is correct - but -a implies -r, so it's redundant to have both flags

Just crontab -e and add 0 0 * * * bash -c 'rsync -a --delete /media/1TB /media/Backup' to the bottom of it. You don't need to have a separate shell script.
Edited by gonX - 10/6/11 at 12:34pm
#well
(19 items)
 
Lenovo L530
(8 items)
 
 
CPUMotherboardGraphicsRAM
Intel Core i7 4770k Gigabyte Z87X-UD4H XFX Radeon HD 6950 Corsair CMX8GX3M2A2000C9 
Hard DriveHard DriveOptical DriveCooling
Samsung EVO 840 Hitachi HDS722020ALA330 Generic DVD±RW Burner Noctua NH-D14 
OSMonitorMonitorKeyboard
Windows 8.1 Dell U2711 LG W2453 Ducky DK9008 Overclock.net Edition, Cherry MX B... 
PowerCaseMouseMouse Pad
Corsair TX850 Antec Three Hundred Corsair Raptor M40 QPAD HeatoN M 
AudioAudioAudio
E-MU Tracker|pre Beyerdynamic DT-770 250 Ohm AntLion ModMIc 
CPUGraphicsRAMRAM
Intel Ivy Bridge 3210M Intel HD 4000 Graphics Soldered Corsair Vengeance  
Hard DriveOptical DriveOSMonitor
500GB DVD-+RW Windows 7 Professional 1600x900 
  hide details  
Reply
#well
(19 items)
 
Lenovo L530
(8 items)
 
 
CPUMotherboardGraphicsRAM
Intel Core i7 4770k Gigabyte Z87X-UD4H XFX Radeon HD 6950 Corsair CMX8GX3M2A2000C9 
Hard DriveHard DriveOptical DriveCooling
Samsung EVO 840 Hitachi HDS722020ALA330 Generic DVD±RW Burner Noctua NH-D14 
OSMonitorMonitorKeyboard
Windows 8.1 Dell U2711 LG W2453 Ducky DK9008 Overclock.net Edition, Cherry MX B... 
PowerCaseMouseMouse Pad
Corsair TX850 Antec Three Hundred Corsair Raptor M40 QPAD HeatoN M 
AudioAudioAudio
E-MU Tracker|pre Beyerdynamic DT-770 250 Ohm AntLion ModMIc 
CPUGraphicsRAMRAM
Intel Ivy Bridge 3210M Intel HD 4000 Graphics Soldered Corsair Vengeance  
Hard DriveOptical DriveOSMonitor
500GB DVD-+RW Windows 7 Professional 1600x900 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Linux, Unix
Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Help setting up a fake raid?