Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Linux download script
New Posts  All Forums:Forum Nav:

Linux download script

post #1 of 9
Thread Starter 
Hey all,

I have 1 network drive and a NAS at home and I would like to download the content from the network drive to my NAS. I usually do this by running a wget command on the NAS. but this isnt ideally because the wget creates dirs which I dont want to have.
This is the code I am using.
Code:
wget -r --ftp-user=admin --ftp-password=admin ftp://192.168.2.2/public/Media/Fotos/*
But this creates the folders 192.168.2.2, public, Media and Fotos. The downloaded content will be download to "/192.168.2.2/public/Media/Fotos/"
So I just want to have the folders who are inside Fotos to be downloaded. so the path will be at end /MyFolder/<Subfolder of Fotos>.

I would also like to know if it is possible to download only those files who arent yet on the NAS?

I hope I explained it clear enough for you to help me.

Thanks in advance

-Rex
Server
(10 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 920 D0 ASUS P6T SE AMD X1600 OCZ Platinum XTC 6GB DDR3-1333 CL7 triple kit 
Hard DriveHard DriveOSMonitor
Maxtor 250GB Samsung 1.5TB HD154UI unRAID v5.0-rc5 Acer H223HQ 
PowerCase
Zalman ZM750-HP 750W Cooler Master Silencio 550 
  hide details  
Reply
Server
(10 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 920 D0 ASUS P6T SE AMD X1600 OCZ Platinum XTC 6GB DDR3-1333 CL7 triple kit 
Hard DriveHard DriveOSMonitor
Maxtor 250GB Samsung 1.5TB HD154UI unRAID v5.0-rc5 Acer H223HQ 
PowerCase
Zalman ZM750-HP 750W Cooler Master Silencio 550 
  hide details  
Reply
post #2 of 9
This might do what you want:

Code:
wget -rnH --ftp-user=admin --ftp-password=admin ftp://192.168.2.2/public/Media/Fotos/*
-nH (no host directories) will cut off everything up to Fotos, and I think Fotos as well, but I'm not absolutely sure about that.
Edited by randomizer - 1/14/11 at 5:19pm
    
CPUMotherboardGraphicsRAM
i7 920 D0 MSI X58 Pro-E GTX 560 Ti 448 3x2GB G.Skill DDR3-1333 9-9-9-24 
Hard DriveHard DriveOptical DriveOS
840 Pro Caviar Black LG BD-ROM Windows 8.1 Pro x64 
MonitorMonitorKeyboardPower
Dell U2713HM Dell U2311H Turbo-Trak (Google it :D) Corsair HX-520 
CaseMouseMouse PadAudio
CM690 Mionix Avior 7000 Everglide Titan AKG K 242 HD 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
i7 920 D0 MSI X58 Pro-E GTX 560 Ti 448 3x2GB G.Skill DDR3-1333 9-9-9-24 
Hard DriveHard DriveOptical DriveOS
840 Pro Caviar Black LG BD-ROM Windows 8.1 Pro x64 
MonitorMonitorKeyboardPower
Dell U2713HM Dell U2311H Turbo-Trak (Google it :D) Corsair HX-520 
CaseMouseMouse PadAudio
CM690 Mionix Avior 7000 Everglide Titan AKG K 242 HD 
  hide details  
Reply
post #3 of 9
I was trying to download some content from a website every hour. I used cron to make this task automatic, I don't think this creates a director and it lets you do what you what so say you need to download to this directory then you can tell it to move the file, rename it. I am sure you can do a lot with this. Here is a link I found maybe this can help ya out

http://www.cyberciti.biz/faq/how-do-...-or-unix-oses/
Main rig
(13 items)
 
  
CPUMotherboardGraphicsRAM
Icore7 2600k P8P67LE ASUS EVGA 660ti 8 gb Corsair Veng 
Hard DriveOSMonitorKeyboard
Silicon Power ssd 120 gb Windows 10 Acer P237HL microshaft ECR 4000 
PowerCaseMouse
ULTRA LSP 650 thermailtake element t Razor Naga  
  hide details  
Reply
Main rig
(13 items)
 
  
CPUMotherboardGraphicsRAM
Icore7 2600k P8P67LE ASUS EVGA 660ti 8 gb Corsair Veng 
Hard DriveOSMonitorKeyboard
Silicon Power ssd 120 gb Windows 10 Acer P237HL microshaft ECR 4000 
PowerCaseMouse
ULTRA LSP 650 thermailtake element t Razor Naga  
  hide details  
Reply
post #4 of 9
Thread Starter 
Quote:
Originally Posted by behappy View Post
I was trying to download some content from a website every hour. I used cron to make this task automatic, I don't think this creates a director and it lets you do what you what so say you need to download to this directory then you can tell it to move the file, rename it. I am sure you can do a lot with this. Here is a link I found maybe this can help ya out

http://www.cyberciti.biz/faq/how-do-...-or-unix-oses/
Thanks for point this peace of software out. I shall try it sometime today!


Quote:
Originally Posted by randomizer View Post
This might do what you want:

Code:
wget -rnH --ftp-user=admin --ftp-password=admin ftp://192.168.2.2/public/Media/Fotos/*
-nH (no host directories) will cut off everything up to Fotos, and I think Fotos as well, but I'm not absolutely sure about that.

I just tried your wget method but this will only leave out the 192.168.2.2 folder it will still create the folders, Public, media and fotos. .


-Rex
Server
(10 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 920 D0 ASUS P6T SE AMD X1600 OCZ Platinum XTC 6GB DDR3-1333 CL7 triple kit 
Hard DriveHard DriveOSMonitor
Maxtor 250GB Samsung 1.5TB HD154UI unRAID v5.0-rc5 Acer H223HQ 
PowerCase
Zalman ZM750-HP 750W Cooler Master Silencio 550 
  hide details  
Reply
Server
(10 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 920 D0 ASUS P6T SE AMD X1600 OCZ Platinum XTC 6GB DDR3-1333 CL7 triple kit 
Hard DriveHard DriveOSMonitor
Maxtor 250GB Samsung 1.5TB HD154UI unRAID v5.0-rc5 Acer H223HQ 
PowerCase
Zalman ZM750-HP 750W Cooler Master Silencio 550 
  hide details  
Reply
post #5 of 9
You could try using curl instead of wget, if you've fully read the manpage and can't seem to get things right.
post #6 of 9
Thread Starter 
Hye,

So I got it all working.. I need to add something more behind the -nH option to leave out the other dirs. here is my command now:
wget -rnH --cut-dirs=3 --ftp-user=admin --ftp-password=admin ftp://192.168.2.2/public/Media/Fotos/*

This will download all the subfolder folder from Fotos. and save them

Thanks for the replys!
Server
(10 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 920 D0 ASUS P6T SE AMD X1600 OCZ Platinum XTC 6GB DDR3-1333 CL7 triple kit 
Hard DriveHard DriveOSMonitor
Maxtor 250GB Samsung 1.5TB HD154UI unRAID v5.0-rc5 Acer H223HQ 
PowerCase
Zalman ZM750-HP 750W Cooler Master Silencio 550 
  hide details  
Reply
Server
(10 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 920 D0 ASUS P6T SE AMD X1600 OCZ Platinum XTC 6GB DDR3-1333 CL7 triple kit 
Hard DriveHard DriveOSMonitor
Maxtor 250GB Samsung 1.5TB HD154UI unRAID v5.0-rc5 Acer H223HQ 
PowerCase
Zalman ZM750-HP 750W Cooler Master Silencio 550 
  hide details  
Reply
post #7 of 9
Nice Based on the logic of that last flag, you could probably omit the -nH and just set --cut-dirs=4. It's probably not going to make any real difference though.
    
CPUMotherboardGraphicsRAM
i7 920 D0 MSI X58 Pro-E GTX 560 Ti 448 3x2GB G.Skill DDR3-1333 9-9-9-24 
Hard DriveHard DriveOptical DriveOS
840 Pro Caviar Black LG BD-ROM Windows 8.1 Pro x64 
MonitorMonitorKeyboardPower
Dell U2713HM Dell U2311H Turbo-Trak (Google it :D) Corsair HX-520 
CaseMouseMouse PadAudio
CM690 Mionix Avior 7000 Everglide Titan AKG K 242 HD 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
i7 920 D0 MSI X58 Pro-E GTX 560 Ti 448 3x2GB G.Skill DDR3-1333 9-9-9-24 
Hard DriveHard DriveOptical DriveOS
840 Pro Caviar Black LG BD-ROM Windows 8.1 Pro x64 
MonitorMonitorKeyboardPower
Dell U2713HM Dell U2311H Turbo-Trak (Google it :D) Corsair HX-520 
CaseMouseMouse PadAudio
CM690 Mionix Avior 7000 Everglide Titan AKG K 242 HD 
  hide details  
Reply
post #8 of 9
If everything is on the LAN, then why not mount the shares using NFS, then sync the directories using rsync? I like luckybackup. It's a really nice front end to rsync. I use it to sync my Documents folders between my laptop, server, and desktop. I mount the servers shares to both the laptop and desktop, then sync.
PWNzershreck
(15 items)
 
  
CPUMotherboardGraphicsRAM
4930K @ 4.6 GHz ASUS Rampage IV Black Edition MSI GTX 1080 FE Heatkiller Acetal 16 GB Corsair Vengeance 1600C9 
Hard DriveOptical DriveCoolingOS
2x Samsung 840 Pro  ASUS DVD-RW SATA Koolance 380i & 2x HW Labs 480GTX Arch Linux x86_64, Windows 7 x64 
MonitorKeyboardPowerCase
LG UC88-B Ultrawide, ASUS VS278Q Ducky Corsair AX1200i Caselabs STH10 
MouseMouse PadAudio
Logitech G500 Func 1030 ASUS Xonar Essence STX 
  hide details  
Reply
PWNzershreck
(15 items)
 
  
CPUMotherboardGraphicsRAM
4930K @ 4.6 GHz ASUS Rampage IV Black Edition MSI GTX 1080 FE Heatkiller Acetal 16 GB Corsair Vengeance 1600C9 
Hard DriveOptical DriveCoolingOS
2x Samsung 840 Pro  ASUS DVD-RW SATA Koolance 380i & 2x HW Labs 480GTX Arch Linux x86_64, Windows 7 x64 
MonitorKeyboardPowerCase
LG UC88-B Ultrawide, ASUS VS278Q Ducky Corsair AX1200i Caselabs STH10 
MouseMouse PadAudio
Logitech G500 Func 1030 ASUS Xonar Essence STX 
  hide details  
Reply
post #9 of 9
Thread Starter 
Quote:
Originally Posted by binormalkilla View Post
If everything is on the LAN, then why not mount the shares using NFS, then sync the directories using rsync? I like luckybackup. It's a really nice front end to rsync. I use it to sync my Documents folders between my laptop, server, and desktop. I mount the servers shares to both the laptop and desktop, then sync.

I shall that try that once I have more time Might take me a afternoon or 2 to get it working XD

-Rex
Server
(10 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 920 D0 ASUS P6T SE AMD X1600 OCZ Platinum XTC 6GB DDR3-1333 CL7 triple kit 
Hard DriveHard DriveOSMonitor
Maxtor 250GB Samsung 1.5TB HD154UI unRAID v5.0-rc5 Acer H223HQ 
PowerCase
Zalman ZM750-HP 750W Cooler Master Silencio 550 
  hide details  
Reply
Server
(10 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 920 D0 ASUS P6T SE AMD X1600 OCZ Platinum XTC 6GB DDR3-1333 CL7 triple kit 
Hard DriveHard DriveOSMonitor
Maxtor 250GB Samsung 1.5TB HD154UI unRAID v5.0-rc5 Acer H223HQ 
PowerCase
Zalman ZM750-HP 750W Cooler Master Silencio 550 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Linux, Unix
Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Linux download script