How to: GPU2 folding with Linux - Overclock.net

error10 · 01-03-09

If running Windows is threatening to put you in a mental institution, maybe you want to run Linux instead, to save what's left of your sanity. Or maybe you want to run Linux to take advantage of the mad SMP PPD it gets but you don't want to sacrifice your GPU folding.

This step-by-step guide will show you how to set up your Linux system to fold on your NVIDIA GPU(s). Sorry, ATi GPUs are not supported at this time.

This guide is for Ubuntu. For other Linux distributions please see post #2.

Prepare your hardware
1. Shut down your computer.
2. If you are currently using SLI you will need to disable it. Remove your SLI bridge.
3. Install any NVIDIA graphics cards you want to use for folding. You do not need monitors or dummy plugs beyond your primary monitor.
4. Start your computer.
Install Ubuntu 64-bit Edition

Using 64-bit Linux is strongly recommended because it will also allow you to run the SMP folding client as well as enable performance gains in most Linux applications, and break the 4GB address space barrier.

If you already have 64-bit Linux you can skip this step. To find out if you have 64-bit Linux:
1. Open a Terminal by going to Applications > Accessories > Terminal.
2. Type uname -m and press Enter.
  Code:
```
uname -m
```
If you see "x86_64" then you have 64-bit Linux. Otherwise, download and install 64-bit Linux by doing the following:
1. Visit the Ubuntu download page.
2. Choose a download location near you from the drop-down list.
3. Scroll down to Custom Options and choose 64bit version.
4. Click Begin Download.
Users with RAID arrays should click "Text based alternate installer" to obtain the 64-bit alternate CD instead.

Once the download is complete, burn the image to CD, reboot with the CD in the drive, and install Ubuntu. Once installation is complete, continue with this guide.
Enable software repositories and update system

WARNING: Your system must be up-to-date before continuing.
1. Go to System > Administration > Software Sources.
2. Ensure that all of the sources are selected, then click Close.
3. Click Reload to update the software repository information.
4. Go to System > Administration > Update Manager. Click Install Updates. If you are prompted to restart your computer, do so.
5. Go to Applications > Accessories > Terminal. Install the packages necessary for compiling programs by typing in the following command.
  Code:
```
sudo apt-get install build-essential
```
Install NVIDIA CUDA drivers

If you have already installed NVIDIA drivers on your system, you will need to replace them with NVIDIA's CUDA-enabled drivers and toolkit, because the drivers which Ubuntu installs do not include CUDA support. Follow these directions to replace your drivers:
1. Download NVIDIA CUDA drivers and toolkit
  1. Visit the NVIDIA CUDA download page.
  2. From the Operating System drop-down, choose Linux 64-bit.
  3. Now scroll down to the CUDA 2.0 section and click the Download button next to CUDA driver: NVIDIA Driver for Linux 64-bit with CUDA Support (177.73). Do NOT use the CUDA 2.1 (180.60) driver; it is very crashy.
  4. A new tab will open. On the new tab, right-click the download link and choose Save Link As.
  5. Click Save. Then close the newly opened Firefox tab with the download link.
  6. Change the Operating System drop-down to Linux 32-bit, then for Linux Version select Ubuntu 8.04 (yes, even if you have 8.10). Now scroll down to the CUDA 2.1 section and click the Download button next to CUDA toolkit: CUDA Toolkit for Ubuntu 8.04.
  7. A new tab will open. On the new tab, right-click the download link and choose Save Link As.
  8. Click Save. Then close the newly opened Firefox tab with the download link.
2. Remove old NVIDIA drivers
  
  WARNING: If you used a third party tool such as Envy to install your NVIDIA drivers, use that same tool to uninstall the drivers now, then skip to the next section.
  
  Go to System > Administration > Hardware Drivers. If the NVIDIA driver is active, click Deactivate, then restart the computer. Return to this screen to ensure that the NVIDIA driver is not active.
3. Install NVIDIA CUDA drivers and toolkit
  
  NOTE: Whenever you receive a new kernel update, you must repeat the steps in this section to update the NVIDIA driver for the new kernel.
  1. The NVIDIA driver will not install successfully by default. In order to install the driver, a minor change must be made to a kernel header file. Go to Applications > Accessories > Terminal and type the following command to edit the necessary file.
    Code:
```
sudo gedit /lib/modules/`uname -r`/build/include/linux/autoconf.h
```
  2. Click Find on the toolbar. Enter the following text, and then click the Find button in the dialog box.
    Code:
```
#define CONFIG_XEN 1
```
  3. Click Close in the dialog box. Then place two forward slashes at the beginning of the highlighted line, without deleting the selected text.
  4. Click Save on the toolbar. Leave the editor window open; we will come back to it shortly.
  5. Return to your Terminal and execute the NVIDIA driver installer.
    Code:
```
sudo sh ~/Desktop/NVIDIA*.run --no-x-check
```
  6. Accept the license agreement. Use the Tab key to highlight the Accept button in white, then press Enter to accept.
  7. Select No; there is no precompiled kernel interface anyway, so there's no point in checking on the Internet for it.
  8. The installer will build the NVIDIA kernel module, then ask if you want to install 32-bit OpenGL libraries. You will need these, so select Yes.
  9. Select Yes to reconfigure the X server to use the new NVIDIA driver.
  10. Driver installation is complete. Select OK.
  11. Return to the editor window containing autoconf.h. Undo your changes by going to Edit > Undo, or just manually removing the two forward slashes.
  12. Save the file and close the editor window. This restores the file to its original condition.
    
    (If you have just reinstalled the driver due to a kernel update, you can stop here. Restart the computer to enable the newly built driver.)
  13. In your Terminal, execute the CUDA toolkit installer. When you are prompted for the installation path, just press Enter.
    Code:
```
sudo sh ~/Desktop/cuda*.run
```
  14. CUDA will be installed on your system.
  15. The system must be configured so that it can locate the CUDA binaries and libraries. Execute the following commands in your Terminal.
    Code:
```
sudo sh -c "echo PATH=/usr/local/cuda/bin:$PATH > /etc/profile.d/cuda.sh"
sudo sh -c "echo /usr/local/cuda/lib > /etc/ld.so.conf.d/cuda.conf"
sudo ldconfig
```
  16. Restart your computer. Go to System > Shut Down, then click Restart.
  17. After the computer restarts, log in and open a Terminal by going to Applications > Accessories > Terminal. Verify that the NVIDIA drivers are running by executing the following command.
    Code:
```
glxinfo | grep vendor
```
    If you do not see NVIDIA Corporation displayed, as above, then run the following command.
    Code:
```
sudo nvidia-xconfig
```
    After running this command, go to System > Log Out, and then log back in. If the NVIDIA Corporation still does not display, the driver is not configured correctly. Seek help before continuing.
Install Wine

Wine is that magical program which lets you run many Windows programs on your Linux computer. In this case, we will use Wine to run the Folding@Home GPU2 client. To install Wine, and prepare it for use by Folding@Home, follow these steps:
1. Go to Applications > Accessories > Terminal. Execute the following command to install Wine.
  Code:
```
sudo apt-get install wine
```
2. Start Applications > Wine > Configure Wine. It may take up to two minutes to start the first time you run it. Be patient.
3. Once Wine Configuration appears, change the Windows Version displayed at the bottom to Windows XP. Then click the Audio tab at the top.
4. When you click the Audio tab, a dialog box will appear saying that defaults have been chosen for your sound card. The defaults are fine, so click OK on the dialog box. Then click OK on the Wine Configuration box to close it. (If you know what you're doing and want to make any changes in your new Wine configuration, you can do so now.)
5. Install the Wine CUDA DLL nvcuda.dll by executing the following command. This DLL helps Wine translate Windows CUDA system calls into Linux CUDA system calls.
  Code:
```
sudo wget http://www.gpu2.twomurs.com/wrapper2ndgen/2.1/cudart.dll.so -O /usr/lib32/wine/nvcuda.dll.so
```
6. Last, we will make a link in the virtual Windows System directory for nvcuda.dll. Execute the following command in your Terminal.
  Code:
```
ln -s /usr/lib32/wine/nvcuda.dll.so ~/.wine/drive_c/windows/system32/nvcuda.dll
```
Install Folding@Home

Finally, we get to the good part! Don't worry, we're in the home stretch now.
1. Using Firefox, download the Folding@Home GPU2 Console client for Windows XP from the Folding@Home High Performance Clients download page. Make sure you do NOT download the systray client, or the client for Vista. Right-click on the Windows icon, and choose Save Link As.
2. Save the file to your desktop.
3. Go to Applications > Accessories > Terminal. Execute the following command to unpack the Folding@Home client. This command will make one copy of the client in a separate directory for each GPU you have installed. The directories will be called nvidia0, nvidia1, etc.
  Code:
```
cd; for i in /dev/nvidia?; do mkdir -p `basename $i`; unzip ~/Desktop/Folding*.zip -d `basename $i`; done
```
4. We will set up one GPU at a time. Execute the following command to change to the nvidia0 directory and run Folding@Home in configuration mode. To set up additional GPUs later, e.g. for the second GPU change nvidia0 to nvidia1 and -gpu 0 to -gpu 1, for the third GPU change nvidia0 to nvidia2 and -gpu 0 to -gpu 2, etc. I suggest opening a new Terminal for each GPU during this stage.
  Code:
```
cd ~/nvidia0
wine Folding@home-Win32-GPU.exe -forcegpu nvidia_g80 -gpu 0 -configonly
```
  I have used the following configuration. Note that I have lowered the amount of memory it reports so that it won't use too many resources, and that I have placed the necessary startup options -forcegpu nvidia_g80 -gpu 0 in. For the second and third GPUs, use -gpu 1, -gpu 2, etc. Also be sure to use a unique Machine ID for each instance. I have started at 3, since I plan to use two SMP clients on this machine and they will have machine ID 1 and 2. For the second and third GPUs, then, use machine IDs 4, 5, etc.
  Code:
```
User name [Anonymous]? error10
Team Number [0]? 37726
Passkey []? (enter your own passkey if you have one)
Ask before fetching/sending work (no/yes) [no]? 
Use proxy (yes/no) [no]? 
Acceptable size of work assignment and work result packets (bigger units
 may have large memory demands) -- 'small' is <5MB, 'normal' is <10MB, and
 'big' is >10MB (small/normal/big) [normal]? 
Change advanced options (yes/no) [no]? yes
Core Priority (idle/low) [idle]? low
CPU usage requested (5-100) [100]? 
Disable highly optimized assembly code (no/yes) [no]? 
Pause if battery power is being used (useful for laptops) (no/yes) [no]? 
Interval, in minutes, between checkpoints (3-30) [15]? 
Memory, in MB, to indicate (3895 available) [3895]? 256
Set -advmethods flag always, requesting new advanced
  scientific cores and/or work units if available (no/yes) [no]? yes
Ignore any deadline information (mainly useful if
 system clock frequently has errors) (no/yes) [no]? 
Machine ID (1-16) [2]? 3
Launch automatically, install as a service in this directory (yes/no) [no]? 
The following options require you to restart the client before they take effect
Disable CPU affinity lock (no/yes) [no]? yes
Additional client parameters []? -forcegpu nvidia_g80 -gpu 0
IP address to bind core to (for viewer) []?
```
5. Now it's time to test it! We'll run it with the -oneunit flag to process a single unit, just to make sure it's working, before we do the final setup on it. Execute the following command.
  Code:
```
wine Folding@home-Win32-GPU.exe -oneunit
```
6. If all goes well, it should download FahCore_11.exe and a work unit and start going. Here I have opened a new Terminal and run the top command to see how much CPU it's consuming. (This is totally optional.) As you can see, it's using no more than 11% of a CPU core, which is pretty good since I had my Q9550 at stock when I made this.
7. In a couple of hours, the single unit should finish, as shown below. If you have multiple GPUs, go back and open a new Terminal to configure each of them now, being sure to change the directory, -gpu option, and machine ID for each one, as explained above.
Configure FahMon

While you're waiting for those first units to fold, it's time to set up FahMon. If you haven't already installed it, open a new Terminal and follow the excellent directions here. Then come back here and we'll set it up to monitor your GPU2 clients.
1. Go to Applications > System Tools > FahMon.
2. On the Clients menu, click Add a new client. Type a name for the client, then click the Browse button (it contains ...). Browse to your nvidia0 directory, then click Open.
3. Now click OK.
4. Repeat the above steps for each of your GPUs after you have configured and started them with -oneunit. Now you should see the status of your running clients.
Final Setup

Finally, I've written a short script which will start all your instances of Folding@home for you, assuming you've followed the directions above precisely. You'll find it attached at the end of this post. Download it to your desktop, and install it with the following command:
Code:
```
sudo install -m755 ~/Desktop/fahgpu.sh.txt /usr/local/bin/fahgpu
```
Now, you can start Folding@home on all of your GPUs at once simply by opening a Terminal and running:
Code:
```
fahgpu
```
(Wait for all of your GPUs to complete their -oneunit, or stop them by pressing Ctrl-C in their respective Terminals, before running the above command.)

Now I've spent all afternoon on this so I'm going to wrap it up here, even though there are probably a few more things I could add. Hopefully this is enough to start anybody GPU2 folding on Linux!

error10 · 01-03-09

Reserved for Fedora guide.

error10 · 01-03-09

Reserved for future updates.

dcshoejake · 01-03-09

you sir will have rep, this is AWESOME!! *JaKe*

kaivorth · 01-03-09

Any PPD boost from doing it within Windows?

error10 · 01-03-09

Quote:

Originally Posted by kaivorth

Any PPD boost from doing it within Windows?

You lose about 10% of the PPD over GPU2 folding in Windows, but you gain a LOT by (1) SMP folding in Linux; (2) running Linux generally. Some of us are aware that running Windows is hazardous to your mental health, after all...

Pic0liter · 01-03-09

AWESOME work error10, I appreciate your dedication to the OCN community, the Folding community, and the Linux community. +

for sure!

**gibsonnova74** · 01-04-09

One of the best guides I have seen! Thanks for the time and effort, I have put it in the Essentials Thread. rep+

error10 · 01-04-09

Quote:

Originally Posted by gibsonnova74

One of the best guides I have seen! Thanks for the time and effort, I have put it in the Essentials Thread. rep+

Thanks. While I was putting it together I ran into a 38 image per post limit! I had to cut out some images.

Mostly less important ones, but still.

And, 38?

sabermetrics · 01-04-09

Wow, this is definitely extremely helpful... a couple months ago, I was trying to find a copy of Vista because I though you couldn't GPU2 fold on Linux...

error10, you sir, are a genius...

		Overclock.net - Overclocking.net > Overclock.net Forum > Overclock.net Folding@Home Team
How to: GPU2 folding with Linux

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)