Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Compiling and CPU usage
New Posts  All Forums:Forum Nav:

Compiling and CPU usage

post #1 of 13
Thread Starter 
In terms of compile times which should have the greatest benefit? Clock speed, number of cores and threads or memory?

I ask because while compiling packages I will nearly run out of memory ( this one only has single channel 4gb) but CPU usage sits at 20% on my i3 @ 4.32. I modified the make.conf to use all threads but it seems to have no effect on compile times.
XEON POWER
(17 items)
 
  
CPUMotherboardGraphicsRAM
Xeon E3-1280 V2 Asus p8h77-i EVGA GTX 570  Corsair Dominator Platinum 1866 cl9 16gb 2x8gb 
Hard DriveHard DriveHard DriveOptical Drive
Samsung Spinpoint F3 Samsung 830 Crucial C300 Samsung USB Blu-Ray 
OSMonitorMonitorKeyboard
Windows 8 Dell U2711 HP 2709m Razer Blackwidow 
PowerCaseMouseMouse Pad
Silverstone Strider Essentials ST60ES-F Bitfenix Prodigy Razer Naga Molten razer sphex 
Audio
onboard 
  hide details  
Reply
XEON POWER
(17 items)
 
  
CPUMotherboardGraphicsRAM
Xeon E3-1280 V2 Asus p8h77-i EVGA GTX 570  Corsair Dominator Platinum 1866 cl9 16gb 2x8gb 
Hard DriveHard DriveHard DriveOptical Drive
Samsung Spinpoint F3 Samsung 830 Crucial C300 Samsung USB Blu-Ray 
OSMonitorMonitorKeyboard
Windows 8 Dell U2711 HP 2709m Razer Blackwidow 
PowerCaseMouseMouse Pad
Silverstone Strider Essentials ST60ES-F Bitfenix Prodigy Razer Naga Molten razer sphex 
Audio
onboard 
  hide details  
Reply
post #2 of 13
The make.conf? On Windows 7? Maybe you should update your sig rig. Or at least mention what distribution you're using.

You need both CPU and RAM to speed up a compile, but you also need disk. Consider the workflow: Load up a source code file, compile it, write the object code back to disk.

Finally, if you really want to know if your changes are having an effect, use the 'time' command, e.g.:

Code:
time make
time emerge -u world
Underground
(14 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 920 C0 ASUS P6T6 WS Revolution GTX 460 TR3X6G1600C8D 
Hard DriveOptical DriveCoolingOS
WD1001FALS SAMSUNG SH-S223F 22X DVD MULTI Corsair H50 Fedora 16 KDE x86_64 
MonitorKeyboardPowerCase
HP w19b Microsoft Comfort Curve Corsair CX600 Thermaltake Armor VA8003BWS 
MouseMouse Pad
Razer DeathAdder Black 
  hide details  
Reply
Underground
(14 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 920 C0 ASUS P6T6 WS Revolution GTX 460 TR3X6G1600C8D 
Hard DriveOptical DriveCoolingOS
WD1001FALS SAMSUNG SH-S223F 22X DVD MULTI Corsair H50 Fedora 16 KDE x86_64 
MonitorKeyboardPowerCase
HP w19b Microsoft Comfort Curve Corsair CX600 Thermaltake Armor VA8003BWS 
MouseMouse Pad
Razer DeathAdder Black 
  hide details  
Reply
post #3 of 13
Thread Starter 
Quote:
Originally Posted by error10 View Post
The make.conf? On Windows 7? Maybe you should update your sig rig. Or at least mention what distribution you're using.
Ha! Sorry it's on my linux box running Arch. It has i3-560 @ 4.35ghz 1x4gb of memory at 1780 9-9-9-24 1T. I was wondering if maybe it was the memory bogging it down since it's only single channel. I would have gone dual channel but I already had the one sitck and I was trying to keep the budget low, could add a second forcheap. The cpu clearly isn't working too hard so it doesn't seem like thats the bottleneck.

I seem to get unusually high memory usage in arch on this thing.
Edited by cavallino - 4/22/11 at 12:18pm
XEON POWER
(17 items)
 
  
CPUMotherboardGraphicsRAM
Xeon E3-1280 V2 Asus p8h77-i EVGA GTX 570  Corsair Dominator Platinum 1866 cl9 16gb 2x8gb 
Hard DriveHard DriveHard DriveOptical Drive
Samsung Spinpoint F3 Samsung 830 Crucial C300 Samsung USB Blu-Ray 
OSMonitorMonitorKeyboard
Windows 8 Dell U2711 HP 2709m Razer Blackwidow 
PowerCaseMouseMouse Pad
Silverstone Strider Essentials ST60ES-F Bitfenix Prodigy Razer Naga Molten razer sphex 
Audio
onboard 
  hide details  
Reply
XEON POWER
(17 items)
 
  
CPUMotherboardGraphicsRAM
Xeon E3-1280 V2 Asus p8h77-i EVGA GTX 570  Corsair Dominator Platinum 1866 cl9 16gb 2x8gb 
Hard DriveHard DriveHard DriveOptical Drive
Samsung Spinpoint F3 Samsung 830 Crucial C300 Samsung USB Blu-Ray 
OSMonitorMonitorKeyboard
Windows 8 Dell U2711 HP 2709m Razer Blackwidow 
PowerCaseMouseMouse Pad
Silverstone Strider Essentials ST60ES-F Bitfenix Prodigy Razer Naga Molten razer sphex 
Audio
onboard 
  hide details  
Reply
post #4 of 13
more ram never hurts, sounds like your processor is just fine... disk write speed will always hold you back though unless you want to jump on the SSD ship
Kinda meh now...
(13 items)
 
  
CPUMotherboardGraphicsRAM
PhII 940 BE asus M4N82 Deluxe 2x 8800gts(g92) 2x 1g ocz ddr2 1066 
Hard DriveOptical DriveOSMonitor
/dev/sd[abc] /dev/sr0 WinXP/Linux 22in acer 
PowerCaseMouseMouse Pad
950w rosewill timebomb Antec 900 (2small) Voodoo Deathadder (RED!) X-Trac Pads Ripper XL 
  hide details  
Reply
Kinda meh now...
(13 items)
 
  
CPUMotherboardGraphicsRAM
PhII 940 BE asus M4N82 Deluxe 2x 8800gts(g92) 2x 1g ocz ddr2 1066 
Hard DriveOptical DriveOSMonitor
/dev/sd[abc] /dev/sr0 WinXP/Linux 22in acer 
PowerCaseMouseMouse Pad
950w rosewill timebomb Antec 900 (2small) Voodoo Deathadder (RED!) X-Trac Pads Ripper XL 
  hide details  
Reply
post #5 of 13
There have been threads in the Gentoo forums comparing package compile times on RAM versus HDD. There isn't much of a real world difference (a few seconds of difference in a 10 min compile). Still it helps to reduce disk wear especially on SSD.
post #6 of 13
I wish my laptop compiled anything in 10 minutes. Take 1-20 hours to compile something hehe
    
CPUMotherboardGraphicsRAM
Athlon II x4 620 2.6GHz (OC 3.2GHz) MSI 790GX-G65 Diamond Radeon 4650 4GB patriot 
Hard DriveOptical DriveOSMonitor
Western Digital ATA 500GB Sony DRU-842A Windows 7 / Fedora 14 / FreeBSD 8.2 ViewSonic VP 201s 
KeyboardPowerCaseMouse
Logitech G15 Antec 650 Watt Antec 900 Logitech G5 
Mouse Pad
A piece of lined paper 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Athlon II x4 620 2.6GHz (OC 3.2GHz) MSI 790GX-G65 Diamond Radeon 4650 4GB patriot 
Hard DriveOptical DriveOSMonitor
Western Digital ATA 500GB Sony DRU-842A Windows 7 / Fedora 14 / FreeBSD 8.2 ViewSonic VP 201s 
KeyboardPowerCaseMouse
Logitech G15 Antec 650 Watt Antec 900 Logitech G5 
Mouse Pad
A piece of lined paper 
  hide details  
Reply
post #7 of 13
Quote:
Originally Posted by cavallino View Post
In terms of compile times which should have the greatest benefit? Clock speed, number of cores and threads or memory?

I ask because while compiling packages I will nearly run out of memory ( this one only has single channel 4gb) but CPU usage sits at 20% on my i3 @ 4.32. I modified the make.conf to use all threads but it seems to have no effect on compile times.
Can you post the relevant contents of /etc/makepkg.conf? A simple error in your syntax might be the issue. In addition, are all threads being lightly loaded or are 1-2 threads being heavily loaded and the rest idle? If the former, it might just be that this particular compilation is so IO heavy that the CPU is left waiting.
    
CPUMotherboardGraphicsRAM
i7 920 D0 MSI X58 Pro-E GTX 560 Ti 448 3x2GB G.Skill DDR3-1333 9-9-9-24 
Hard DriveHard DriveOptical DriveOS
840 Pro Caviar Black LG BD-ROM Windows 8.1 Pro x64 
MonitorMonitorKeyboardPower
Dell U2713HM Dell U2311H Turbo-Trak (Google it :D) Corsair HX-520 
CaseMouseMouse PadAudio
CM690 Mionix Avior 7000 Everglide Titan AKG K 242 HD 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
i7 920 D0 MSI X58 Pro-E GTX 560 Ti 448 3x2GB G.Skill DDR3-1333 9-9-9-24 
Hard DriveHard DriveOptical DriveOS
840 Pro Caviar Black LG BD-ROM Windows 8.1 Pro x64 
MonitorMonitorKeyboardPower
Dell U2713HM Dell U2311H Turbo-Trak (Google it :D) Corsair HX-520 
CaseMouseMouse PadAudio
CM690 Mionix Avior 7000 Everglide Titan AKG K 242 HD 
  hide details  
Reply
post #8 of 13
it really depends on what you're compiling. if the code has a lot of independent components that can take advantage of something like make -j 4 or something, then more cores the better (but that will load up on memory and disk I/O too). some programs are not written that way, so you can't compile several parts in parallel so more cores becomes pointless. some code take a lot of memory to compile, or perhaps you're using some compile time optimization that requires a lot of memory especially if you're unrolling a highly recursive function or something like that....

so, "it depends" ....
TAIPEI
(10 items)
 
AURORA
(13 items)
 
 
MotherboardGraphicsRAMHard Drive
ASRock X99 Extreme11 EVGA GTX 980 Superclocked 32GB 8x4GB Corsair LPX Samsung XP941  
Hard DriveCoolingOSMonitor
Western Digital 3TB RE Noctua NH-D15 Fedora 21 Linux Samsung S27D590C 
PowerCase
Seasonic SS-1200XP Cooler Master Cosmos II 
CPUMotherboardGraphicsRAM
Dual Quad-core L5430 2.66Ghz 12mb cache Intel 5000 chipset ATI ES1000 64GB FBDIMM DDR2 PC2-5300 667Mhz 
Hard DriveOSPower
WD3000FYYZ PERC H700 w/ 512MB cache CentOS 7.2.1511 950W x2 
  hide details  
Reply
TAIPEI
(10 items)
 
AURORA
(13 items)
 
 
MotherboardGraphicsRAMHard Drive
ASRock X99 Extreme11 EVGA GTX 980 Superclocked 32GB 8x4GB Corsair LPX Samsung XP941  
Hard DriveCoolingOSMonitor
Western Digital 3TB RE Noctua NH-D15 Fedora 21 Linux Samsung S27D590C 
PowerCase
Seasonic SS-1200XP Cooler Master Cosmos II 
CPUMotherboardGraphicsRAM
Dual Quad-core L5430 2.66Ghz 12mb cache Intel 5000 chipset ATI ES1000 64GB FBDIMM DDR2 PC2-5300 667Mhz 
Hard DriveOSPower
WD3000FYYZ PERC H700 w/ 512MB cache CentOS 7.2.1511 950W x2 
  hide details  
Reply
post #9 of 13
Thread Starter 
Quote:
Originally Posted by randomizer View Post
Can you post the relevant contents of /etc/makepkg.conf? A simple error in your syntax might be the issue. In addition, are all threads being lightly loaded or are 1-2 threads being heavily loaded and the rest idle? If the former, it might just be that this particular compilation is so IO heavy that the CPU is left waiting.
Ok this was it i'd made a typo with an extra space on the makeflag line. :Facepalm:

Now I have it set to -j5 (I read in the arch wiki to make it number of threads/cores +1) and now I'm getting much higher cpu usage upe to 100% and hugely faster compile times.
Edited by cavallino - 4/23/11 at 9:53am
XEON POWER
(17 items)
 
  
CPUMotherboardGraphicsRAM
Xeon E3-1280 V2 Asus p8h77-i EVGA GTX 570  Corsair Dominator Platinum 1866 cl9 16gb 2x8gb 
Hard DriveHard DriveHard DriveOptical Drive
Samsung Spinpoint F3 Samsung 830 Crucial C300 Samsung USB Blu-Ray 
OSMonitorMonitorKeyboard
Windows 8 Dell U2711 HP 2709m Razer Blackwidow 
PowerCaseMouseMouse Pad
Silverstone Strider Essentials ST60ES-F Bitfenix Prodigy Razer Naga Molten razer sphex 
Audio
onboard 
  hide details  
Reply
XEON POWER
(17 items)
 
  
CPUMotherboardGraphicsRAM
Xeon E3-1280 V2 Asus p8h77-i EVGA GTX 570  Corsair Dominator Platinum 1866 cl9 16gb 2x8gb 
Hard DriveHard DriveHard DriveOptical Drive
Samsung Spinpoint F3 Samsung 830 Crucial C300 Samsung USB Blu-Ray 
OSMonitorMonitorKeyboard
Windows 8 Dell U2711 HP 2709m Razer Blackwidow 
PowerCaseMouseMouse Pad
Silverstone Strider Essentials ST60ES-F Bitfenix Prodigy Razer Naga Molten razer sphex 
Audio
onboard 
  hide details  
Reply
post #10 of 13
-march=native -O2 -pipe
-j5

for i7

those + ssd = really fast compile times.

with my old e6300/ddr2/hd emerge -e world used to take longer than overnight. Now with new build and even more packages it is about 2 hrs.

ssd ftw
stable again
(25 items)
 
  
CPUCPUMotherboardGraphics
E5-2687W E5-2687W ASUS Z9PED8-WS EVGA GTX 570 (Linux host) 
GraphicsRAMHard DriveHard Drive
EVGA GTX 970 FTW (win7 guest) 64GB G.SKILL 2133 2x Crucial M4 256GB raid1 4x 3TB raid 10 
CoolingCoolingCoolingCooling
2x Apogee HD  2x RX 480 2x MCP 655 RP-452x2 rev2 (new) 
CoolingCoolingOSOS
16x Cougar Turbine CFT12SB4 (new) EK FC 580 Gentoo (host) Gentoo (x23 guests) 
OSMonitorMonitorPower
windows 7 (guest w/ vfio-pci) Viewsonic 23" 1080P Viewsonic 19" Antec HCP Platinum 1000 (new) 
CaseOtherOther
Case Labs TH10 (still the best ever) 2x Lamptron FC-5 IOGEAR 2 way DVI KVM Switch 
  hide details  
Reply
stable again
(25 items)
 
  
CPUCPUMotherboardGraphics
E5-2687W E5-2687W ASUS Z9PED8-WS EVGA GTX 570 (Linux host) 
GraphicsRAMHard DriveHard Drive
EVGA GTX 970 FTW (win7 guest) 64GB G.SKILL 2133 2x Crucial M4 256GB raid1 4x 3TB raid 10 
CoolingCoolingCoolingCooling
2x Apogee HD  2x RX 480 2x MCP 655 RP-452x2 rev2 (new) 
CoolingCoolingOSOS
16x Cougar Turbine CFT12SB4 (new) EK FC 580 Gentoo (host) Gentoo (x23 guests) 
OSMonitorMonitorPower
windows 7 (guest w/ vfio-pci) Viewsonic 23" 1080P Viewsonic 19" Antec HCP Platinum 1000 (new) 
CaseOtherOther
Case Labs TH10 (still the best ever) 2x Lamptron FC-5 IOGEAR 2 way DVI KVM Switch 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Linux, Unix
Overclock.net › Forums › Software, Programming and Coding › Operating Systems › Linux, Unix › Compiling and CPU usage