Overclock.net banner

[Linux] GPU2 folding

32165 Views 146 Replies 35 Participants Last post by  zodac
If running Windows is threatening to put you in a mental institution, maybe you want to run Linux instead, to save what's left of your sanity. Or maybe you want to run Linux to take advantage of the mad SMP PPD it gets but you don't want to sacrifice your GPU folding.

This step-by-step guide will show you how to set up your Linux system to fold on your NVIDIA GPU(s). Sorry, ATi GPUs are not supported at this time.

This guide is for Ubuntu. For other Linux distributions please see post #2.
  1. Prepare your hardware
    1. Shut down your computer.
    2. If you are currently using SLI you will need to disable it. Remove your SLI bridge.
    3. Install any NVIDIA graphics cards you want to use for folding. You do not need monitors or dummy plugs beyond your primary monitor.
    4. Start your computer.
  2. Install Ubuntu 64-bit Edition

    Using 64-bit Linux is strongly recommended because it will also allow you to run the SMP folding client as well as enable performance gains in most Linux applications, and break the 4GB address space barrier.

    If you already have 64-bit Linux you can skip this step. To find out if you have 64-bit Linux:
    1. Open a Terminal by going to Applications > Accessories > Terminal.
    2. Type uname -m and press Enter.

      Code:

      Code:
      uname -m
    If you see "x86_64" then you have 64-bit Linux. Otherwise, download and install 64-bit Linux by doing the following:
    1. Visit the Ubuntu download page.
    2. Choose a Download Location near you from the drop-down list.
    3. Click Alternative download options to expand the available options.
    4. Under Choose the architecture choose 64bit version.
    5. Click Begin Download.
    Users with RAID arrays should click "Text based alternate installer" to obtain the 64-bit alternate CD instead.

    Once the download is complete, burn the image to CD, reboot with the CD in the drive, and install Ubuntu. Once installation is complete, continue with this guide.
  3. Enable software repositories and update system

    WARNING: Your system must be up-to-date before continuing.
    1. Go to System > Administration > Software Sources.
    2. Ensure that all of the sources are selected, then click Close.

    3. Click Reload to update the software repository information.

    4. Go to System > Administration > Update Manager. Click Install Updates. If you are prompted to restart your computer, do so.

    5. Go to Applications > Accessories > Terminal. Install the packages necessary for compiling programs by typing in the following command.

      Code:

      Code:
      sudo apt-get install build-essential
  4. Install NVIDIA CUDA drivers

    If you have already installed NVIDIA drivers on your system, you will need to replace them with NVIDIA's CUDA-enabled drivers and toolkit, because the drivers which Ubuntu installs do not include CUDA support. Follow these directions to replace your drivers:
    1. Download NVIDIA CUDA drivers and toolkit
      1. Visit the NVIDIA CUDA download page.
      2. From the Operating System drop-down, choose Linux 64-bit.
      3. Now scroll down to the CUDA 2.0 section and click the Download button next to CUDA driver: NVIDIA Driver for Linux 64-bit with CUDA Support (177.73). Do NOT use the CUDA 2.1 (180.60) driver; it is very crashy.

      4. A new tab will open. On the new tab, right-click the download link and choose Save Link As.
      5. Click Save. Then close the newly opened Firefox tab with the download link.

      6. Change the Operating System drop-down to Linux 32-bit, then for Linux Version select Ubuntu 8.04 (yes, even if you have 8.10). Now scroll down to the CUDA 2.1 section and click the Download button next to CUDA toolkit: CUDA Toolkit for Ubuntu 8.04.

      7. A new tab will open. On the new tab, right-click the download link and choose Save Link As.
      8. Click Save. Then close the newly opened Firefox tab with the download link.
    2. Remove old NVIDIA drivers

      WARNING: If you used a third party tool such as Envy to install your NVIDIA drivers, use that same tool to uninstall the drivers now, then skip to the next section.

      Go to System > Administration > Hardware Drivers. If the NVIDIA driver is active, click Deactivate, then restart the computer. Return to this screen to ensure that the NVIDIA driver is not active.

    3. Install NVIDIA CUDA drivers and toolkit

      NOTE: Whenever you receive a new kernel update, you must repeat the steps in this section to update the NVIDIA driver for the new kernel.
      1. The NVIDIA driver will not install successfully by default. In order to install the driver, a minor change must be made to a kernel header file. Go to Applications > Accessories > Terminal and type the following command to edit the necessary file.

        Code:

        Code:
        sudo gedit /lib/modules/`uname -r`/build/include/linux/autoconf.h
      2. Click Find on the toolbar. Enter the following text, and then click the Find button in the dialog box.

        Code:

        Code:
        #define CONFIG_XEN 1

      3. Click Close in the dialog box. Then place two forward slashes at the beginning of the highlighted line, without deleting the selected text.

      4. Click Save on the toolbar. Leave the editor window open; we will come back to it shortly.
      5. Return to your Terminal and execute the NVIDIA driver installer.

        Code:

        Code:
        sudo sh ~/Desktop/NVIDIA*.run --no-x-check
      6. Accept the license agreement. Use the Tab key to highlight the Accept button in white, then press Enter to accept.

      7. Select No; there is no precompiled kernel interface anyway, so there's no point in checking on the Internet for it.

      8. The installer will build the NVIDIA kernel module, then ask if you want to install 32-bit OpenGL libraries. You will need these, so select Yes.

      9. Select Yes to reconfigure the X server to use the new NVIDIA driver.

      10. Driver installation is complete. Select OK.
      11. Return to the editor window containing autoconf.h. Undo your changes by going to Edit > Undo, or just manually removing the two forward slashes.

      12. Save the file and close the editor window. This restores the file to its original condition.


        (If you have just reinstalled the driver due to a kernel update, you can stop here. Restart the computer to enable the newly built driver.)
      13. In your Terminal, execute the CUDA toolkit installer. When you are prompted for the installation path, just press Enter.

        Code:

        Code:
        sudo sh ~/Desktop/cuda*.run

      14. CUDA will be installed on your system.

      15. The system must be configured so that it can locate the CUDA binaries and libraries. Execute the following commands in your Terminal.

        Code:

        Code:
        sudo sh -c "echo PATH=/usr/local/cuda/bin:$PATH > /etc/profile.d/cuda.sh"
        sudo sh -c "echo /usr/local/cuda/lib > /etc/ld.so.conf.d/cuda.conf"
        sudo ldconfig

      16. Restart your computer. Go to System > Shut Down, then click Restart.

      17. After the computer restarts, log in and open a Terminal by going to Applications > Accessories > Terminal. Verify that the NVIDIA drivers are running by executing the following command.

        Code:

        Code:
        glxinfo | grep vendor


        If you do not see NVIDIA Corporation displayed, as above, then run the following command.

        Code:

        Code:
        sudo nvidia-xconfig
        After running this command, go to System > Log Out, and then log back in. If the NVIDIA Corporation still does not display, the driver is not configured correctly. Seek help before continuing.
  5. Install Wine

    Wine is that magical program which lets you run many Windows programs on your Linux computer. In this case, we will use Wine to run the [email protected] GPU2 client. To install Wine, and prepare it for use by [email protected], follow these steps:
    1. Go to Applications > Accessories > Terminal. Execute the following command to install Wine.

      Code:

      Code:
      sudo apt-get install wine
    2. Start Applications > Wine > Configure Wine. It may take up to two minutes to start the first time you run it. Be patient.
    3. Once Wine Configuration appears, change the Windows Version displayed at the bottom to Windows XP. Then click the Audio tab at the top.

    4. When you click the Audio tab, a dialog box will appear saying that defaults have been chosen for your sound card. The defaults are fine, so click OK on the dialog box. Then click OK on the Wine Configuration box to close it. (If you know what you're doing and want to make any changes in your new Wine configuration, you can do so now.)

    5. Install the Wine CUDA DLL nvcuda.dll by executing the following command. This DLL helps Wine translate Windows CUDA system calls into Linux CUDA system calls.

      Code:

      Code:
      sudo wget http://www.gpu2.twomurs.com/wrapper2ndgen/2.1/cudart.dll.so -O /usr/lib32/wine/nvcuda.dll.so

    6. Last, we will make a link in the virtual Windows System directory for nvcuda.dll. Execute the following command in your Terminal.

      Code:

      Code:
      ln -s /usr/lib32/wine/nvcuda.dll.so ~/.wine/drive_c/windows/system32/nvcuda.dll
  6. Install [email protected]

    Finally, we get to the good part! Don't worry, we're in the home stretch now.
    1. Using Firefox, download the [email protected] GPU2 Console client for Windows XP from the [email protected] High Performance Clients download page. Make sure you do NOT download the systray client, or the client for Vista. Right-click on the Windows icon, and choose Save Link As.

    2. Save the file to your desktop.

    3. Go to Applications > Accessories > Terminal. Execute the following command to unpack the [email protected] client. This command will make one copy of the client in a separate directory for each GPU you have installed. The directories will be called nvidia0, nvidia1, etc.

      Code:

      Code:
      cd; for i in /dev/nvidia?; do mkdir -p `basename $i`; unzip ~/Desktop/Folding*.zip -d `basename $i`; done

    4. We will set up one GPU at a time. Execute the following command to change to the nvidia0 directory and run [email protected] in configuration mode. To set up additional GPUs later, e.g. for the second GPU change nvidia0 to nvidia1 and -gpu 0 to -gpu 1, for the third GPU change nvidia0 to nvidia2 and -gpu 0 to -gpu 2, etc. I suggest opening a new Terminal for each GPU during this stage.

      Code:

      Code:
      cd ~/nvidia0
      wine [email protected] -forcegpu nvidia_g80 -gpu 0 -configonly
      I have used the following configuration. Note that I have lowered the amount of memory it reports so that it won't use too many resources, and that I have placed the necessary startup options -forcegpu nvidia_g80 -gpu 0 in. For the second and third GPUs, use -gpu 1, -gpu 2, etc. Also be sure to use a unique Machine ID for each instance. I have started at 3, since I plan to use two SMP clients on this machine and they will have machine ID 1 and 2. For the second and third GPUs, then, use machine IDs 4, 5, etc.

      Code:

      Code:
      User name [Anonymous]? [B]error10[/B]
      Team Number [0]? [B]37726[/B]
      Passkey []? [B](enter your own passkey if you have one)[/B]
      Ask before fetching/sending work (no/yes) [no]? 
      Use proxy (yes/no) [no]? 
      Acceptable size of work assignment and work result packets (bigger units
       may have large memory demands) -- 'small' is <5MB, 'normal' is <10MB, and
       'big' is >10MB (small/normal/big) [normal]? 
      Change advanced options (yes/no) [no]? [B]yes[/B]
      Core Priority (idle/low) [idle]? [B]low[/B]
      CPU usage requested (5-100) [100]? 
      Disable highly optimized assembly code (no/yes) [no]? 
      Pause if battery power is being used (useful for laptops) (no/yes) [no]? 
      Interval, in minutes, between checkpoints (3-30) [15]? 
      Memory, in MB, to indicate (3895 available) [3895]? [B]256[/B]
      Set -advmethods flag always, requesting new advanced
        scientific cores and/or work units if available (no/yes) [no]? [B]yes[/B]
      Ignore any deadline information (mainly useful if
       system clock frequently has errors) (no/yes) [no]? 
      Machine ID (1-16) [2]? [B]3[/B]
      Launch automatically, install as a service in this directory (yes/no) [no]? 
      The following options require you to restart the client before they take effect
      Disable CPU affinity lock (no/yes) [no]? [B]yes[/B]
      Additional client parameters []? [B]-forcegpu nvidia_g80 -gpu 0[/B]
      IP address to bind core to (for viewer) []?

    5. Now it's time to test it! We'll run it with the -oneunit flag to process a single unit, just to make sure it's working, before we do the final setup on it. Execute the following command.

      Code:

      Code:
      wine [email protected] -oneunit
    6. If all goes well, it should download FahCore_11.exe and a work unit and start going. Here I have opened a new Terminal and run the top command to see how much CPU it's consuming. (This is totally optional.) As you can see, it's using no more than 11% of a CPU core, which is pretty good since I had my Q9550 at stock when I made this.

    7. In a couple of hours, the single unit should finish, as shown below. If you have multiple GPUs, go back and open a new Terminal to configure each of them now, being sure to change the directory, -gpu option, and machine ID for each one, as explained above.
  7. Configure FahMon

    While you're waiting for those first units to fold, it's time to set up FahMon. If you haven't already installed it, open a new Terminal and follow the excellent directions here. Then come back here and we'll set it up to monitor your GPU2 clients.
    1. Go to Applications > System Tools > FahMon.
    2. On the Clients menu, click Add a new client. Type a name for the client, then click the Browse button (it contains ...). Browse to your nvidia0 directory, then click Open.

    3. Now click OK.

    4. Repeat the above steps for each of your GPUs after you have configured and started them with -oneunit. Now you should see the status of your running clients.
  8. Final Setup

    Finally, I've written a short script which will start all your instances of [email protected] for you, assuming you've followed the directions above precisely. You'll find it attached at the end of this post. Download it to your desktop, and install it with the following command:

    Code:

    Code:
    sudo install -m755 ~/Desktop/fahgpu.sh.txt /usr/local/bin/fahgpu
    Now, you can start [email protected] on all of your GPUs at once simply by opening a Terminal and running:

    Code:

    Code:
    fahgpu
    (Wait for all of your GPUs to complete their -oneunit, or stop them by pressing Ctrl-C in their respective Terminals, before running the above command.)

    Now I've spent all afternoon on this so I'm going to wrap it up here, even though there are probably a few more things I could add. Hopefully this is enough to start anybody GPU2 folding on Linux!
1 - 20 of 147 Posts

· Premium Member
Joined
·
14,563 Posts
Discussion Starter · #2 ·
Reserved for Fedora guide.
 

· Premium Member
Joined
·
14,563 Posts
Discussion Starter · #3 ·
Reserved for future updates.
 

· Premium Member
Joined
·
7,493 Posts
Any PPD boost from doing it within Windows?
 

· Premium Member
Joined
·
14,563 Posts
Discussion Starter · #6 ·
Quote:

Originally Posted by kaivorth View Post
Any PPD boost from doing it within Windows?
You lose about 10% of the PPD over GPU2 folding in Windows, but you gain a LOT by (1) SMP folding in Linux; (2) running Linux generally. Some of us are aware that running Windows is hazardous to your mental health, after all...
 

· Premium Member
Joined
·
14,563 Posts
Discussion Starter · #9 ·
Quote:

Originally Posted by gibsonnova74 View Post
One of the best guides I have seen! Thanks for the time and effort, I have put it in the Essentials Thread. rep+
Thanks. While I was putting it together I ran into a 38 image per post limit! I had to cut out some images.
Mostly less important ones, but still.

And, 38?
 

· Premium Member
Joined
·
14,563 Posts
Discussion Starter · #11 ·
Updated the OP with info on dummy plugs. Dummy plugs are NOT required.
 

· Premium Member
Joined
·
5,959 Posts
Awesome guide. However one day I hope we will not require one of these just to GPU fold under linux. I'd honestly do it but I don't know if I have the patience to get it up and running.

+rep regardless of me using it
 

· Premium Member
Joined
·
14,563 Posts
Discussion Starter · #13 ·
Quote:

Originally Posted by Microsis View Post
Awesome guide. However one day I hope we will not require one of these just to GPU fold under linux. I'd honestly do it but I don't know if I have the patience to get it up and running.

+rep regardless of me using it

The first time is the hardest. I can set it up in my sleep now.
In fact I'm going to do it again from scratch this afternoon!
 

· Premium Member
Joined
·
14,563 Posts
Discussion Starter · #15 ·
Quote:

Originally Posted by Greg2008uk View Post
Thanks for the great guide, I can go back to Ubuntu now!!! +rep

Do you get a higher PPD over vmware and ubuntu?
I don't run Windows if I can avoid it. It's how I stay sane.
 

· Premium Member
Joined
·
14,563 Posts
Discussion Starter · #17 ·
Quote:


Originally Posted by Greg2008uk
View Post

Do you only need one smp client running, will it use all four cores?

Go here.
 

· Premium Member
Joined
·
14,563 Posts
Discussion Starter · #18 ·
Well I've somehow managed to kill a brand new 4GB USB stick putting this guide together, while working on part 2 tonight. I hope OCZ RMA doesn't completely suck.
 

· Banned
Joined
·
942 Posts
Quote:


Originally Posted by error10
View Post

I don't run Windows if I can avoid it. It's how I stay sane.


i'm just the opposite... my short day in ubuntu led me to believe that there was absolutely no reason for me to use it... ever.

except if i wanted to keep trying to figure out how do download and install software that's needed to download and install other software


and my experience with the SMP on linux is that there was a 100 minute advantage (using linux over windows) over a 24 hour period on the same WU and same credit for said WU - at least according to FAHMON
 

· Premium Member
Joined
·
14,563 Posts
Discussion Starter · #20 ·
Quote:


Originally Posted by pixie
View Post

i'm just the opposite... my short day in ubuntu led me to believe that there was absolutely no reason for me to use it... ever.

except if i wanted to keep trying to figure out how do download and install software that's needed to download and install other software


and my experience with the SMP on linux is that there was a 100 minute advantage (using linux over windows) over a 24 hour period on the same WU and same credit for said WU - at least according to FAHMON

Unfair. After your first short day with Windows you were still installing drivers and all those little utilities Microsoft should include but doesn't.

Sounds like you didn't click the right button to download the software you wanted. The problem you had doesn't sound anywhere near typical.

Anyway.
 
1 - 20 of 147 Posts
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top