Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Sacrifice Compression Size for Speed, non-GUI
New Posts  All Forums:Forum Nav:

Sacrifice Compression Size for Speed, non-GUI

post #1 of 3
Thread Starter 
Hey guys.

I have just received a challenge that I need some help with on Linux. At work, we constantly are compressing files. My job is to figure out how to take a 400MB+ sparse file, and compress it to less than or equal to 20MB in 30 seconds or less. Currently, it takes 90 seconds to compress to ~4MB.

What command line tools (along with what options) will allow me to adjust the "ratio" between time and space?
Orion IV
(12 items)
 
Supernaut II
(8 items)
 
 
CPUMotherboardGraphicsRAM
Core i7 3930k Gigabyte GA-X79-UD3 2x EVGA GTX 770 SC in SLI 16GB (4x4GB) G.SKILL Sniper 1600 
Hard DriveCoolingOSMonitor
ADATA 128GB SSD + 2x WD Blue 500GB AAKX (RAID 0) Thermaltake Water2.0 Extreme Dual Rad | 10x Yat... Windows 7 Ultimate x64 ASUS PB278Q, ASUS VG248QE, 2x Dell 20" 16:10 
KeyboardPowerCaseMouse
Two. Browns & Blues LEPA G900 (900W) CoolerMaster HAF 932 Advanced Logitech G700 
CPUMotherboardGraphicsRAM
AMD A6 6400k Black Edition Gigabyte F2A85XM-D3H Integrated 8GB Kingston HyperX 
Hard DriveOSPowerCase
4x 2TB WD Green (RAID 5), 80GB OS Windows 7 Ultimate 400W Generic Thermaltake V3 Black 
CPUMotherboardGraphicsRAM
Core i5 750 ASUS P55 XFX 5750 8GB Kingston 
Hard DriveOSPowerCase
160GB WD Blue Windows 7 Ultimate 600w FEMA Diablotek Short-Midtower 
  hide details  
Reply
Orion IV
(12 items)
 
Supernaut II
(8 items)
 
 
CPUMotherboardGraphicsRAM
Core i7 3930k Gigabyte GA-X79-UD3 2x EVGA GTX 770 SC in SLI 16GB (4x4GB) G.SKILL Sniper 1600 
Hard DriveCoolingOSMonitor
ADATA 128GB SSD + 2x WD Blue 500GB AAKX (RAID 0) Thermaltake Water2.0 Extreme Dual Rad | 10x Yat... Windows 7 Ultimate x64 ASUS PB278Q, ASUS VG248QE, 2x Dell 20" 16:10 
KeyboardPowerCaseMouse
Two. Browns & Blues LEPA G900 (900W) CoolerMaster HAF 932 Advanced Logitech G700 
CPUMotherboardGraphicsRAM
AMD A6 6400k Black Edition Gigabyte F2A85XM-D3H Integrated 8GB Kingston HyperX 
Hard DriveOSPowerCase
4x 2TB WD Green (RAID 5), 80GB OS Windows 7 Ultimate 400W Generic Thermaltake V3 Black 
CPUMotherboardGraphicsRAM
Core i5 750 ASUS P55 XFX 5750 8GB Kingston 
Hard DriveOSPowerCase
160GB WD Blue Windows 7 Ultimate 600w FEMA Diablotek Short-Midtower 
  hide details  
Reply
post #2 of 3
This is the site I use when I need to remember what a commandline switch is for 7zip. It uses .exe as an example, but I don't see why the linux version of 7zip would be different...
Aoi
(20 items)
 
Midori
(14 items)
 
 
CPUMotherboardGraphicsRAM
Core i7 920 D0 Gigabyte G1.Killer Guerilla GTX670 4GB SLI 24GB Corsair Vengeance 
Hard DriveHard DriveOptical DriveCooling
WD Velociraptor Samsung F1 Blu-ray XL Corsair H70 
OSMonitorMonitorMonitor
Windows 7 Professional x64 Dell 2405FPW Dell U2410 Dell 2405FPW 
MonitorKeyboardPowerCase
Dell U2311H Microsoft Sidewinder X4 Silverstone Strider 1kw Corsair 700D 
MouseMouse PadAudioOther
Logitech G500 Ozone XL Integrated Logitech G13 
CPUMotherboardGraphicsRAM
Core i5 3570K Asus P8Z77-M Pro nVidia GTX680 Corsair Vengeance LP 16GB 
Hard DriveOptical DriveOSOS
WD Velociraptor 600GB Samsung DVD+RW Windows 7 Home Premium x64 Ubuntu Server Customised 
MonitorKeyboardPowerCase
Triple Dell U2412M Sidewinder X6 Corsair TX750 Fractal Design R4 
Mouse
Logitech G700 
  hide details  
Reply
Aoi
(20 items)
 
Midori
(14 items)
 
 
CPUMotherboardGraphicsRAM
Core i7 920 D0 Gigabyte G1.Killer Guerilla GTX670 4GB SLI 24GB Corsair Vengeance 
Hard DriveHard DriveOptical DriveCooling
WD Velociraptor Samsung F1 Blu-ray XL Corsair H70 
OSMonitorMonitorMonitor
Windows 7 Professional x64 Dell 2405FPW Dell U2410 Dell 2405FPW 
MonitorKeyboardPowerCase
Dell U2311H Microsoft Sidewinder X4 Silverstone Strider 1kw Corsair 700D 
MouseMouse PadAudioOther
Logitech G500 Ozone XL Integrated Logitech G13 
CPUMotherboardGraphicsRAM
Core i5 3570K Asus P8Z77-M Pro nVidia GTX680 Corsair Vengeance LP 16GB 
Hard DriveOptical DriveOSOS
WD Velociraptor 600GB Samsung DVD+RW Windows 7 Home Premium x64 Ubuntu Server Customised 
MonitorKeyboardPowerCase
Triple Dell U2412M Sidewinder X6 Corsair TX750 Fractal Design R4 
Mouse
Logitech G700 
  hide details  
Reply
post #3 of 3
If you want speed, aim for gzip with different compression ratios or lzo/lzop. Just use the -1 to -9 flags, eg. gzip -6 file. lzop uses the same flags.

lzop is a lot faster than gzip, but has a worse compression ratio.
#well
(19 items)
 
Lenovo L530
(8 items)
 
 
CPUMotherboardGraphicsRAM
Intel Core i7 4770k Gigabyte Z87X-UD4H XFX Radeon HD 6950 Corsair CMX8GX3M2A2000C9 
Hard DriveHard DriveOptical DriveCooling
Samsung EVO 840 Hitachi HDS722020ALA330 Generic DVD±RW Burner Noctua NH-D14 
OSMonitorMonitorKeyboard
Windows 8.1 Dell U2711 LG W2453 Ducky DK9008 Overclock.net Edition, Cherry MX B... 
PowerCaseMouseMouse Pad
Corsair TX850 Antec Three Hundred Corsair Raptor M40 QPAD HeatoN M 
AudioAudioAudio
E-MU Tracker|pre Beyerdynamic DT-770 250 Ohm AntLion ModMIc 
CPUGraphicsRAMRAM
Intel Ivy Bridge 3210M Intel HD 4000 Graphics Soldered Corsair Vengeance  
Hard DriveOptical DriveOSMonitor
500GB DVD-+RW Windows 7 Professional 1600x900 
  hide details  
Reply
#well
(19 items)
 
Lenovo L530
(8 items)
 
 
CPUMotherboardGraphicsRAM
Intel Core i7 4770k Gigabyte Z87X-UD4H XFX Radeon HD 6950 Corsair CMX8GX3M2A2000C9 
Hard DriveHard DriveOptical DriveCooling
Samsung EVO 840 Hitachi HDS722020ALA330 Generic DVD±RW Burner Noctua NH-D14 
OSMonitorMonitorKeyboard
Windows 8.1 Dell U2711 LG W2453 Ducky DK9008 Overclock.net Edition, Cherry MX B... 
PowerCaseMouseMouse Pad
Corsair TX850 Antec Three Hundred Corsair Raptor M40 QPAD HeatoN M 
AudioAudioAudio
E-MU Tracker|pre Beyerdynamic DT-770 250 Ohm AntLion ModMIc 
CPUGraphicsRAMRAM
Intel Ivy Bridge 3210M Intel HD 4000 Graphics Soldered Corsair Vengeance  
Hard DriveOptical DriveOSMonitor
500GB DVD-+RW Windows 7 Professional 1600x900 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Linux, Unix
Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Sacrifice Compression Size for Speed, non-GUI