Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › The best to replicate an installation (such as ubuntu server)
New Posts  All Forums:Forum Nav:

The best to replicate an installation (such as ubuntu server)

post #1 of 8
Thread Starter 
I have done some searching but thought the community would have a much better idea/experience with solutions. I should mention I'm still a bit of beginner here. At least that's how I think of myself.

Here's the scenario:

Two 8 gig thumb drives purchased as a set (same manufacturer/model/capacity/i assume batch)
The first one, the "source" thumb drive, has an installation of ubuntu server already on it and can be booted successfully (the default settings for partitions/partition sizes).

The second one, the "destination" drive, is currently blank (the default format, 1 partition with fat32 or whatever)

What I would like to do is replicate the source drive exactly onto the second drive in the easiest/relatively pain free way possible.

Now before you say just use dd to clone the drive over I actually already did that and had less than stellar results (the destination drive was extremely slow and just didn't act quite the same for whatever reason).

I'm hoping for some alternative approaches to doing this. I haven't figure out the quickest way to partition/format the destination drive yet but for the second part of moving all the files over I have imagined these two possible solutions:
  • Use RSYNC to sync all the files over such that everything on the destination is the same size/version/content/hash/etc and if there's anything on the destination not on the source it is deleted...
  • Use the tar+gzip combo to compress the contents of each partition on the source, extracting to the destination (this would have the added benefit of regular gzipped backups of the source)
  • Perhaps some combination of the two


I would also back up the boot sector of the source (actual device names/paths, not with standing something like dd if=/dev/sb1 of=/home/subassy/bootsect.img count=1 bs=512) and restore to the destination just to make the replication as close as possible.


In summary, here are my questions:
  1. am I on the right track in replicating an install?
  2. Any super-neat automated ways of replicating the partition lay out and formatting?
  3. Are there any other and/or better ways of doing described above?
 
VM Server
(17 items)
 
 
CPUGraphicsRAMHard Drive
Intel Ivy Bridge Core i7-3630QM nVidia GeForce GTX 680M 16GB DDR3 1600MHz Dual Channel Memory (2 SODIMMS) Hard Drive: Serial-ATA II 3GB/s 
Hard DriveOSMonitorPower
Hard Drive: Serial-ATA II 3GB/s Windows 10 Pro x64 17.3" FHD 16:9 (1920x1080) Battery: Smart Li-ion Battery (8-Cell) 
Audio
Sound Blaster Compatible 3D Audio 
CPUMotherboardGraphicsRAM
Intel Core i7 860 Biostar T5 XE Radeon HD 5870 Corsair 16GB  
Hard DriveHard DriveOptical DriveOS
Western Digital hard drive wd1001fals-00e8b0 Maxtor 300GB I don't need no stinking optical drive Microsoft Windows 7 Ultimate x64 
MonitorMonitorKeyboardPower
HP ZR24w 24'' Samsung SyncMaster 24" logitech wireless k360 Seventeam ST-850ZAF 850W ATX 
CaseMouseAudioAudio
Thermaltake V9 Black Edition Logitech G500 Programmable Gaming Mouse FiiO E7 USB DAC and Portable Headphone Amplifier Sennheiser HD555 Professional Headphones 
  hide details  
Reply
 
VM Server
(17 items)
 
 
CPUGraphicsRAMHard Drive
Intel Ivy Bridge Core i7-3630QM nVidia GeForce GTX 680M 16GB DDR3 1600MHz Dual Channel Memory (2 SODIMMS) Hard Drive: Serial-ATA II 3GB/s 
Hard DriveOSMonitorPower
Hard Drive: Serial-ATA II 3GB/s Windows 10 Pro x64 17.3" FHD 16:9 (1920x1080) Battery: Smart Li-ion Battery (8-Cell) 
Audio
Sound Blaster Compatible 3D Audio 
CPUMotherboardGraphicsRAM
Intel Core i7 860 Biostar T5 XE Radeon HD 5870 Corsair 16GB  
Hard DriveHard DriveOptical DriveOS
Western Digital hard drive wd1001fals-00e8b0 Maxtor 300GB I don't need no stinking optical drive Microsoft Windows 7 Ultimate x64 
MonitorMonitorKeyboardPower
HP ZR24w 24'' Samsung SyncMaster 24" logitech wireless k360 Seventeam ST-850ZAF 850W ATX 
CaseMouseAudioAudio
Thermaltake V9 Black Edition Logitech G500 Programmable Gaming Mouse FiiO E7 USB DAC and Portable Headphone Amplifier Sennheiser HD555 Professional Headphones 
  hide details  
Reply
post #2 of 8
http://forum.proxmox.com/threads/12523-dd-to-ignore-empty-space-of-raw-files

Last post has a good option. Use that image, and write it (probably with DD) to the disc. Once written, you should be able to expand the partition to fit the entire disc, essentially growing it. (growfs?) After that, it should be the exact same.

though if the discs are the same size and you have enough space to store the DD image, DD should work just the same. No matter what you are going to have speed issues.

Another way is to create your own install image. I'd suggest Debian over ubuntu as a server. Once you have your own base image, it'll install the same on any computer.

http://jan.exss.de/debian-cd.html

Make the base system, use that link to have that base system create your install image. Copy image to whatever medium, install and be done! I believe Ubuntu has similar methods, google should procure good examples.
Current Rig
(14 items)
 
  
CPUMotherboardGraphicsRAM
FX-8350 4.6GHz@1.44v GA-990FXA-UD3 R4.0 HD 7950 (1100/1450) 8G Muskin DDR3 1866@8CLS 
Hard DriveOptical DriveOSMonitor
1TB WD LiteOn DVD-RW DL Linux/Windows 19" Phillips TV 1080p 
PowerCaseMouseMouse Pad
OCZ 600W Generic Junk Logitech MX400 Generic Junk 
Audio
SBL 5.1 
  hide details  
Reply
Current Rig
(14 items)
 
  
CPUMotherboardGraphicsRAM
FX-8350 4.6GHz@1.44v GA-990FXA-UD3 R4.0 HD 7950 (1100/1450) 8G Muskin DDR3 1866@8CLS 
Hard DriveOptical DriveOSMonitor
1TB WD LiteOn DVD-RW DL Linux/Windows 19" Phillips TV 1080p 
PowerCaseMouseMouse Pad
OCZ 600W Generic Junk Logitech MX400 Generic Junk 
Audio
SBL 5.1 
  hide details  
Reply
post #3 of 8
The dd install may have ran slower because of the hardware block sizes or something. I think flash drives can be picky like that, but I may be wrong.

Clonezilla is probably your best bet. by default it does file system copies that are more respectful to your destination hardware than a straight dd. (Though clonezilla will fall back on dd if all else fails)
post #4 of 8
Quote:
Originally Posted by Plan9 View Post

The dd install may have ran slower because of the hardware block sizes or something. I think flash drives can be picky like that, but I may be wrong.

Clonezilla is probably your best bet. by default it does file system copies that are more respectful to your destination hardware than a straight dd. (Though clonezilla will fall back on dd if all else fails)

http://linux-howto-guide.blogspot.com/2009/10/increase-usb-flash-drive-write-speed.html Yup! I remember one of my drives suffering from extremely slow writes in which case I had to resort to something similar.
Current Rig
(14 items)
 
  
CPUMotherboardGraphicsRAM
FX-8350 4.6GHz@1.44v GA-990FXA-UD3 R4.0 HD 7950 (1100/1450) 8G Muskin DDR3 1866@8CLS 
Hard DriveOptical DriveOSMonitor
1TB WD LiteOn DVD-RW DL Linux/Windows 19" Phillips TV 1080p 
PowerCaseMouseMouse Pad
OCZ 600W Generic Junk Logitech MX400 Generic Junk 
Audio
SBL 5.1 
  hide details  
Reply
Current Rig
(14 items)
 
  
CPUMotherboardGraphicsRAM
FX-8350 4.6GHz@1.44v GA-990FXA-UD3 R4.0 HD 7950 (1100/1450) 8G Muskin DDR3 1866@8CLS 
Hard DriveOptical DriveOSMonitor
1TB WD LiteOn DVD-RW DL Linux/Windows 19" Phillips TV 1080p 
PowerCaseMouseMouse Pad
OCZ 600W Generic Junk Logitech MX400 Generic Junk 
Audio
SBL 5.1 
  hide details  
Reply
post #5 of 8
Good link. That was kind of along the lines of what I was thinking; manually create the partition table then use clonezilla to copy the contents.
post #6 of 8
Should be able to do that all with a script, no?
Current Rig
(14 items)
 
  
CPUMotherboardGraphicsRAM
FX-8350 4.6GHz@1.44v GA-990FXA-UD3 R4.0 HD 7950 (1100/1450) 8G Muskin DDR3 1866@8CLS 
Hard DriveOptical DriveOSMonitor
1TB WD LiteOn DVD-RW DL Linux/Windows 19" Phillips TV 1080p 
PowerCaseMouseMouse Pad
OCZ 600W Generic Junk Logitech MX400 Generic Junk 
Audio
SBL 5.1 
  hide details  
Reply
Current Rig
(14 items)
 
  
CPUMotherboardGraphicsRAM
FX-8350 4.6GHz@1.44v GA-990FXA-UD3 R4.0 HD 7950 (1100/1450) 8G Muskin DDR3 1866@8CLS 
Hard DriveOptical DriveOSMonitor
1TB WD LiteOn DVD-RW DL Linux/Windows 19" Phillips TV 1080p 
PowerCaseMouseMouse Pad
OCZ 600W Generic Junk Logitech MX400 Generic Junk 
Audio
SBL 5.1 
  hide details  
Reply
post #7 of 8
Quote:
Originally Posted by mushroomboy View Post

Should be able to do that all with a script, no?

Clonezilla is basically just a set of scripts around standard GNU tools. I just think there's little point re-engineering the wheel when Clonezilla already works (plus the Clonezilla snapshot can be used as a backup too)
post #8 of 8
Thread Starter 
Quote:
Originally Posted by mushroomboy View Post

http://forum.proxmox.com/threads/12523-dd-to-ignore-empty-space-of-raw-files

Last post has a good option. Use that image, and write it (probably with DD) to the disc. Once written, you should be able to expand the partition to fit the entire disc, essentially growing it. (growfs?) After that, it should be the exact same.

though if the discs are the same size and you have enough space to store the DD image, DD should work just the same. No matter what you are going to have speed issues.

Another way is to create your own install image. I'd suggest Debian over ubuntu as a server. Once you have your own base image, it'll install the same on any computer.

http://jan.exss.de/debian-cd.html

Make the base system, use that link to have that base system create your install image. Copy image to whatever medium, install and be done! I believe Ubuntu has similar methods, google should procure good examples.

Quote:
Originally Posted by mushroomboy View Post

http://linux-howto-guide.blogspot.com/2009/10/increase-usb-flash-drive-write-speed.html Yup! I remember one of my drives suffering from extremely slow writes in which case I had to resort to something similar.

Quote:
Originally Posted by Plan9 View Post

Clonezilla is basically just a set of scripts around standard GNU tools. I just think there's little point re-engineering the wheel when Clonezilla already works (plus the Clonezilla snapshot can be used as a backup too)

Thanks for all the info guys!

Based what little experience I had with it I thought clonezilla was a straight byte-for-byte imager and thereby...basically a GUI to dd. Didn't know it worked other than that. But I was only imaging NTFS drives so that probably had something to do with it. I'm going to be exploring the clonezilla options as well.

I did some more search and was able to find this sort of solution for replicating the partition structure. I haven't actually tried any of this.
Code:
sfdisk -d /dev/sda > partition.txt
(modify partition.txt for paths/devices as appropriate)
sfdisk /dev/sdb < partition.txt

Followed by creating file system (I think I read ext2 is better for thumb drives but you get the idea), mounting both devices and doing a recursive copy of all the files.
Code:
mkfs -t ext3 /dev/sdb1
mount -t ext3 /dev/sda1 /mnt/source
mount -t ext3 /dev/sdb1 /mnt/destination
Use cp command to copy the data over
cp -a /mnt/source/* /mnt/destination
-a: preserve all attributes , never follow symbolic links and copy recursively

The destination would still need a boot loader I think. I've read about if not actually issued grub commands (just grub-install /dev/sdx or something, right?).

Wouldn't this solution or some variation therein work?
 
VM Server
(17 items)
 
 
CPUGraphicsRAMHard Drive
Intel Ivy Bridge Core i7-3630QM nVidia GeForce GTX 680M 16GB DDR3 1600MHz Dual Channel Memory (2 SODIMMS) Hard Drive: Serial-ATA II 3GB/s 
Hard DriveOSMonitorPower
Hard Drive: Serial-ATA II 3GB/s Windows 10 Pro x64 17.3" FHD 16:9 (1920x1080) Battery: Smart Li-ion Battery (8-Cell) 
Audio
Sound Blaster Compatible 3D Audio 
CPUMotherboardGraphicsRAM
Intel Core i7 860 Biostar T5 XE Radeon HD 5870 Corsair 16GB  
Hard DriveHard DriveOptical DriveOS
Western Digital hard drive wd1001fals-00e8b0 Maxtor 300GB I don't need no stinking optical drive Microsoft Windows 7 Ultimate x64 
MonitorMonitorKeyboardPower
HP ZR24w 24'' Samsung SyncMaster 24" logitech wireless k360 Seventeam ST-850ZAF 850W ATX 
CaseMouseAudioAudio
Thermaltake V9 Black Edition Logitech G500 Programmable Gaming Mouse FiiO E7 USB DAC and Portable Headphone Amplifier Sennheiser HD555 Professional Headphones 
  hide details  
Reply
 
VM Server
(17 items)
 
 
CPUGraphicsRAMHard Drive
Intel Ivy Bridge Core i7-3630QM nVidia GeForce GTX 680M 16GB DDR3 1600MHz Dual Channel Memory (2 SODIMMS) Hard Drive: Serial-ATA II 3GB/s 
Hard DriveOSMonitorPower
Hard Drive: Serial-ATA II 3GB/s Windows 10 Pro x64 17.3" FHD 16:9 (1920x1080) Battery: Smart Li-ion Battery (8-Cell) 
Audio
Sound Blaster Compatible 3D Audio 
CPUMotherboardGraphicsRAM
Intel Core i7 860 Biostar T5 XE Radeon HD 5870 Corsair 16GB  
Hard DriveHard DriveOptical DriveOS
Western Digital hard drive wd1001fals-00e8b0 Maxtor 300GB I don't need no stinking optical drive Microsoft Windows 7 Ultimate x64 
MonitorMonitorKeyboardPower
HP ZR24w 24'' Samsung SyncMaster 24" logitech wireless k360 Seventeam ST-850ZAF 850W ATX 
CaseMouseAudioAudio
Thermaltake V9 Black Edition Logitech G500 Programmable Gaming Mouse FiiO E7 USB DAC and Portable Headphone Amplifier Sennheiser HD555 Professional Headphones 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Linux, Unix
Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › The best to replicate an installation (such as ubuntu server)