Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › [SOLVED] Dynamically allocated squashfs image
New Posts  All Forums:Forum Nav:

[SOLVED] Dynamically allocated squashfs image

post #1 of 4
Thread Starter 
Anybody have any experience with this? I do a lot of data forensics in my off time (which is dwindling these days, I don't even really have the time to do what I'm asking for information about in this thread, but I'll get to it eventually) but I never have to work with dynamicly sized images outside of VM's since a RAW image is superior in every way for forensics.

Basically what I want to do is make a small raw image file, give it a false size (so its a 1TB file, with a size of only a few MB's on disk) and then format it SquashFS. The purpose of this is compressing my raw disk images in a way that I can still directly interface with them, instead of doing dd into a gz so I end up with image.raw.tar.gz and then having to extract it to work with it, I can dd straight into a raw image file located in the mounted squash image, where I can work directly with the image. This way I can condense the space utilized by my client's images. All of the client images are currently housed inside a 1TB Luks encrypted image, which is about 50% free space itself, though its still 1TB on disk. One of the images is of a 160gb drive, but only 15GB of the drive was actually utilized - I still want the raw image, but I want to compress all those zeroes...
Using SquashFS for the entire array isn't really a viable option because I need high performance from the filesystem for other applications.

Edit: I'm a moron. Sparse files should be able to accomplish what I'm looking for.
Edited by Xaero252 - 7/21/13 at 9:12pm
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
post #2 of 4
Thread Starter 
Solution:

1. Create a sparse file container for your compressed filesystem:
(replace 1T in this command for what size you desire, 1T is 1 terabyte, as its sparse it will consume zero bytes on creation)
Code:
$ truncate -s 1T image.img 
2. Chose a filesystem & format the image: (I chose btrfs as LZO compression is pretty optimal for my typical workload)
Code:
# losetup loop0 ./image.img
# mkfs.btrfs /dev/loop0
3. Mount & Use
Code:
# mount -o rw,compress=lzo /dev/loop0 /path/to/directory
So simple, dunno why it wasn't obvious.
Edited by Xaero252 - 7/22/13 at 1:33am
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
post #3 of 4
I've not played around with sparse files before. This is handy to know - so thanks for updating the thread with your solution thumb.gif
post #4 of 4
Thread Starter 
Quote:
Originally Posted by Plan9 View Post

I've not played around with sparse files before. This is handy to know - so thanks for updating the thread with your solution thumb.gif

I don't play with them often, and commonly they aren't the best solution but in this case it seems to be exactly what I wanted. My Luks encrypted image has now been compressed to 135g (since most of it was free space) The only problem with this method is if you remove files... it doesn't magically reclaim space. That said since btrfs supports trim commands I should be able to execute fstrim, and then "sparsify" with the volume unmounted to reclaim space from it, should I need to delete anything. I (re)learn something new every day!
Edited by Xaero252 - 7/22/13 at 10:45am
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Linux, Unix
Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › [SOLVED] Dynamically allocated squashfs image