Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Synchronize a pair of folders?
New Posts  All Forums:Forum Nav:

Synchronize a pair of folders?

post #1 of 15
Thread Starter 
I was wondering if anyone knew a way off hand to synchronize a pair of folders (such as over samba) so that the network copy and the local copy would alway be the same.

the main issue I see with this is large files will be frequently written to both directories, and I'm not sure if this will cause problems such as the linux host (the other folder is on a windows box) will start to copy the file before it is completely copied, or will start deleting files after my windows pc is shut down for the night.

anyone have any experience with this sort of setup?
Renaissance
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 4790K GA-Z97n-Gaming 5 HIS 7850 Mushkin Blackline 2x8GB DDR3 2133 
Hard DriveCoolingOSCase
Corsair GS 240 Corsair H50 Windows 8.1 Lian-li PC-Q08R 
Mouse
Razer Deathadder 3500dpi 
  hide details  
Reply
Renaissance
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 4790K GA-Z97n-Gaming 5 HIS 7850 Mushkin Blackline 2x8GB DDR3 2133 
Hard DriveCoolingOSCase
Corsair GS 240 Corsair H50 Windows 8.1 Lian-li PC-Q08R 
Mouse
Razer Deathadder 3500dpi 
  hide details  
Reply
post #2 of 15
Greetz
To tell you the truth I can't remember if this works over Samba or not but I think thr odds are good. For Linux directories this is exactly what you are looking for.

See this http://en.wikipedia.org/wiki/Rsync

and This

http://everythinglinux.org/rsync/

Rsync is extremely powerful and useful and is well worth any effort you can exert to learn it's "tricks of the trade". The concept is in the first link and the 2nd is a tutorial. Bookmark 'em.
NewMain
(16 items)
 
  
CPUMotherboardGraphicsRAM
Intel i5 - 3550 Asrock Z77 Extreme4 Gigabyte GTX 760  4x2GB Corsair Vengeance 
Hard DriveOptical DriveCoolingOS
Seagate SATA 2TB x 2  Plextor PX-891SAW CM-Hyper N520 Slackware 14, Studio KUbuntu, OpenSuSe 12.3, Wi... 
MonitorKeyboardPowerCase
32" Vizio HDTV + DLP Logitech Wireless Corsair HX-850 Antec Sonata I 
MouseMouse PadAudioOther
Razer DeathAdder 2013 dual ESI Juli@ CoolGear ExtSata Enclosure w/ Optical and 3TB S... 
  hide details  
Reply
NewMain
(16 items)
 
  
CPUMotherboardGraphicsRAM
Intel i5 - 3550 Asrock Z77 Extreme4 Gigabyte GTX 760  4x2GB Corsair Vengeance 
Hard DriveOptical DriveCoolingOS
Seagate SATA 2TB x 2  Plextor PX-891SAW CM-Hyper N520 Slackware 14, Studio KUbuntu, OpenSuSe 12.3, Wi... 
MonitorKeyboardPowerCase
32" Vizio HDTV + DLP Logitech Wireless Corsair HX-850 Antec Sonata I 
MouseMouse PadAudioOther
Razer DeathAdder 2013 dual ESI Juli@ CoolGear ExtSata Enclosure w/ Optical and 3TB S... 
  hide details  
Reply
post #3 of 15
Try http://www.digionica.com/Products. If needed, as a developer can give you a free key
post #4 of 15
Quote:
Originally Posted by Oksnor View Post
Try http://www.digionica.com/Products. If needed, as a developer can give you a free key
Why pay money for a Windows program when rsync is free and solid? Why recommend a Windoze program in a Linux section?
NewMain
(16 items)
 
  
CPUMotherboardGraphicsRAM
Intel i5 - 3550 Asrock Z77 Extreme4 Gigabyte GTX 760  4x2GB Corsair Vengeance 
Hard DriveOptical DriveCoolingOS
Seagate SATA 2TB x 2  Plextor PX-891SAW CM-Hyper N520 Slackware 14, Studio KUbuntu, OpenSuSe 12.3, Wi... 
MonitorKeyboardPowerCase
32" Vizio HDTV + DLP Logitech Wireless Corsair HX-850 Antec Sonata I 
MouseMouse PadAudioOther
Razer DeathAdder 2013 dual ESI Juli@ CoolGear ExtSata Enclosure w/ Optical and 3TB S... 
  hide details  
Reply
NewMain
(16 items)
 
  
CPUMotherboardGraphicsRAM
Intel i5 - 3550 Asrock Z77 Extreme4 Gigabyte GTX 760  4x2GB Corsair Vengeance 
Hard DriveOptical DriveCoolingOS
Seagate SATA 2TB x 2  Plextor PX-891SAW CM-Hyper N520 Slackware 14, Studio KUbuntu, OpenSuSe 12.3, Wi... 
MonitorKeyboardPowerCase
32" Vizio HDTV + DLP Logitech Wireless Corsair HX-850 Antec Sonata I 
MouseMouse PadAudioOther
Razer DeathAdder 2013 dual ESI Juli@ CoolGear ExtSata Enclosure w/ Optical and 3TB S... 
  hide details  
Reply
post #5 of 15
Quote:
Why recommend a Windoze program in a Linux section?
Excuse me. I'll be more attentive in future.
Quote:
Why pay money for a Windows program
It's not advertisement. I'm looking for betatester/feedback. I say:
Quote:
...as a developer can give you a free
Sorry for my English
post #6 of 15
Thread Starter 
just a confirmation about rsync, it will detect when the remote host is shut off and not delete all the files in the mirror folder, right?

and is it capable of detecting that a file is still being written to?
Renaissance
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 4790K GA-Z97n-Gaming 5 HIS 7850 Mushkin Blackline 2x8GB DDR3 2133 
Hard DriveCoolingOSCase
Corsair GS 240 Corsair H50 Windows 8.1 Lian-li PC-Q08R 
Mouse
Razer Deathadder 3500dpi 
  hide details  
Reply
Renaissance
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 4790K GA-Z97n-Gaming 5 HIS 7850 Mushkin Blackline 2x8GB DDR3 2133 
Hard DriveCoolingOSCase
Corsair GS 240 Corsair H50 Windows 8.1 Lian-li PC-Q08R 
Mouse
Razer Deathadder 3500dpi 
  hide details  
Reply
post #7 of 15
Why not give dropbox a try? Free, you can sync a specific folder to both locations, and it works with every OS out there. It also keeps copies on the web. Limited to 2GB if you want free though.
    
CPUMotherboardGraphicsRAM
Phenom IIx4 955 @3.2ghz 1.285V Asus M4A79XTD EVO 6950 @ 850/1250 4x2GB DDR3 
Hard DriveOptical DriveOSMonitor
Hitachi 1Tb + Seagate 1.5Tb Samsung super speed Windows 7 Pro(games)/Linux Mint 10 (not games) LP2065 + 2x 2007FPb in IPS Eyefinity 4800*1200 
KeyboardPowerCaseMouse
Logitech Wave Corsair HX620 Azza Solano 1000 Logitech MX revolution 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Phenom IIx4 955 @3.2ghz 1.285V Asus M4A79XTD EVO 6950 @ 850/1250 4x2GB DDR3 
Hard DriveOptical DriveOSMonitor
Hitachi 1Tb + Seagate 1.5Tb Samsung super speed Windows 7 Pro(games)/Linux Mint 10 (not games) LP2065 + 2x 2007FPb in IPS Eyefinity 4800*1200 
KeyboardPowerCaseMouse
Logitech Wave Corsair HX620 Azza Solano 1000 Logitech MX revolution 
  hide details  
Reply
post #8 of 15
Thread Starter 
2GB is not nearly enough, as it's going to be keeping a mirror of my media library between my personal machine and my server, for redundancy and streaming purposes.
Renaissance
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 4790K GA-Z97n-Gaming 5 HIS 7850 Mushkin Blackline 2x8GB DDR3 2133 
Hard DriveCoolingOSCase
Corsair GS 240 Corsair H50 Windows 8.1 Lian-li PC-Q08R 
Mouse
Razer Deathadder 3500dpi 
  hide details  
Reply
Renaissance
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 4790K GA-Z97n-Gaming 5 HIS 7850 Mushkin Blackline 2x8GB DDR3 2133 
Hard DriveCoolingOSCase
Corsair GS 240 Corsair H50 Windows 8.1 Lian-li PC-Q08R 
Mouse
Razer Deathadder 3500dpi 
  hide details  
Reply
post #9 of 15
Quote:
Originally Posted by ZHoob2004 View Post
just a confirmation about rsync, it will detect when the remote host is shut off and not delete all the files in the mirror folder, right?

and is it capable of detecting that a file is still being written to?
How rsync behaves is totally up to you. Although there are GUI frontends available the core application is command line with an incredible array of switches to customize what and how it synchronizes. It also can run manually whenever you say so or as a daemon more or less continually or controlled by cron. It is even used totally on demand for the purpose of upgrading entire OpSys versions as in rolling releases. It is utterly dependable.

Rsync is so successful that it is the preferred application for enterprise work even when multiple computers, whether Unix/Linux, Windows, or Mac must be synchronized to a master server. It should be obvious that deleting due to simple issues like one being off has to be one of the first safeguards for such mission critical work. I assure you that if this is for a SOHO system it will exceed your expectations. It is simply brilliant and totally a servant to your commands.

Ig you have any reservations even after you have read up on it, simply set up a test case and see for yourself..
NewMain
(16 items)
 
  
CPUMotherboardGraphicsRAM
Intel i5 - 3550 Asrock Z77 Extreme4 Gigabyte GTX 760  4x2GB Corsair Vengeance 
Hard DriveOptical DriveCoolingOS
Seagate SATA 2TB x 2  Plextor PX-891SAW CM-Hyper N520 Slackware 14, Studio KUbuntu, OpenSuSe 12.3, Wi... 
MonitorKeyboardPowerCase
32" Vizio HDTV + DLP Logitech Wireless Corsair HX-850 Antec Sonata I 
MouseMouse PadAudioOther
Razer DeathAdder 2013 dual ESI Juli@ CoolGear ExtSata Enclosure w/ Optical and 3TB S... 
  hide details  
Reply
NewMain
(16 items)
 
  
CPUMotherboardGraphicsRAM
Intel i5 - 3550 Asrock Z77 Extreme4 Gigabyte GTX 760  4x2GB Corsair Vengeance 
Hard DriveOptical DriveCoolingOS
Seagate SATA 2TB x 2  Plextor PX-891SAW CM-Hyper N520 Slackware 14, Studio KUbuntu, OpenSuSe 12.3, Wi... 
MonitorKeyboardPowerCase
32" Vizio HDTV + DLP Logitech Wireless Corsair HX-850 Antec Sonata I 
MouseMouse PadAudioOther
Razer DeathAdder 2013 dual ESI Juli@ CoolGear ExtSata Enclosure w/ Optical and 3TB S... 
  hide details  
Reply
post #10 of 15
Thread Starter 
all right, I'm working on setting up rsync now, and trying to use a windows rsync client (deltacopy in this case) and I keep getting an error

Code:
Executing: rsync.exe  -v -rlt -z --chmod=a=rw,Da+x --delete "/cygdrive/D/Video/" "192.168.0.29::video/Video/"
sending incremental file list

rsync: writefd_unbuffered failed to write 4 bytes [sender]: Software caused connection abort (113)
rsync: read error: Software caused connection abort (113)
rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/rsync-3.0.4-1/src/rsync-3.0.4/io.c(791) [sender=3.0.4]

Error in rsync protocol data stream
Rsync.exe returned an error. Will try again. This is retry number 1 of 5

Executing: rsync.exe  -v -rlt -z --chmod=a=rw,Da+x --delete "/cygdrive/D/Video/" "192.168.0.29::video/Video/"
sending incremental file list

rsync: writefd_unbuffered failed to write 4 bytes [sender]: Software caused connection abort (113)
rsync: read error: Software caused connection abort (113)
rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/rsync-3.0.4-1/src/rsync-3.0.4/io.c(791) [sender=3.0.4]

Error in rsync protocol data stream
Rsync.exe returned an error. Will try again. This is retry number 2 of 5

Executing: rsync.exe  -v -rlt -z --chmod=a=rw,Da+x --delete "/cygdrive/D/Video/" "192.168.0.29::video/Video/"
sending incremental file list

rsync: writefd_unbuffered failed to write 4 bytes [sender]: Software caused connection abort (113)
rsync: read error: Software caused connection abort (113)
rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/rsync-3.0.4-1/src/rsync-3.0.4/io.c(791) [sender=3.0.4]

Error in rsync protocol data stream
Rsync.exe returned an error. Will try again. This is retry number 3 of 5

Executing: rsync.exe  -v -rlt -z --chmod=a=rw,Da+x --delete "/cygdrive/D/Video/" "192.168.0.29::video/Video/"
sending incremental file list

rsync: writefd_unbuffered failed to write 4 bytes [sender]: Software caused connection abort (113)
rsync: read error: Software caused connection abort (113)
rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/rsync-3.0.4-1/src/rsync-3.0.4/io.c(791) [sender=3.0.4]

Error in rsync protocol data stream
Rsync.exe returned an error. Will try again. This is retry number 4 of 5

Executing: rsync.exe  -v -rlt -z --chmod=a=rw,Da+x --delete "/cygdrive/D/Video/" "192.168.0.29::video/Video/"
sending incremental file list

rsync: writefd_unbuffered failed to write 4 bytes [sender]: Software caused connection abort (113)
rsync: read error: Software caused connection abort (113)
rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/rsync-3.0.4-1/src/rsync-3.0.4/io.c(791) [sender=3.0.4]

Error in rsync protocol data stream
Profile 'video' executed in 1360 milliseconds. One or more errors were encountered.
any ideas?
Renaissance
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 4790K GA-Z97n-Gaming 5 HIS 7850 Mushkin Blackline 2x8GB DDR3 2133 
Hard DriveCoolingOSCase
Corsair GS 240 Corsair H50 Windows 8.1 Lian-li PC-Q08R 
Mouse
Razer Deathadder 3500dpi 
  hide details  
Reply
Renaissance
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 4790K GA-Z97n-Gaming 5 HIS 7850 Mushkin Blackline 2x8GB DDR3 2133 
Hard DriveCoolingOSCase
Corsair GS 240 Corsair H50 Windows 8.1 Lian-li PC-Q08R 
Mouse
Razer Deathadder 3500dpi 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Linux, Unix
Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Synchronize a pair of folders?