Overclock.net banner
1 - 20 of 347 Posts

·
Registered
Joined
·
201 Posts
Discussion Starter · #1 ·
Hi,

I'm new to this forum, so if I've posted the wrong place, I'm sorry o)
It's here I found a post/thread related our problem

RACK-mounted Boinc Server - Ubuntu Server 12.04.4 64bit - BOINC - GPU - Non-graphical Environment - No GUI

Attention Mr. "Tex1954 " and Mr. "b3Machi7ke"

I'm building several RACK-mounted Ubuntu Server 12.04.4 64bit only for use with BOINC. "boinc-client" that is. It's a complete non-graphical environment. No GUI what so ever! I remotely control the servers by using Putty/SSH from a Windows 7 rig/and a Ubuntu Desktop as well. Not important!
But the problem is that the GPU of my GeForce GT610 (and GT640) is not recognized by BOINC. So we have been discussing this for more than 3 weeks now at berkeley.edu. No solution to the problem yet!

I searched the net for 2840 sites or more, of course, but the best and the most serious "almost" solution to the problem, I found here.
Done by "Tex1954": http://www.overclock.net/t/1123532/guide-gui-linux-for-boinc-how-i-do-it-done-for-now
This is also a kind of answer to reply #44 in the same thread. b3Machi7ke is asking Tex1954 "...it's possible to run BOINC on CUDA through a terminal interface only, with no GUI?.."

Installing the Ubuntu Server 12.04.4 and then BOINC by "apt-get install boinc-client" and using the repository by Gianfranco too, resulted in the GPU was not recognized by BOINC. When recognized "NVIDIA GeForce GT 610 (134215679MB) OpenCL: 1.01" is showed at a site, where we can see all our computers running boinc "http://setiathome.berkeley.edu/hosts_user.php"
Many of us thought it was because of the nouveau driver. Some thought it was because of X-server needed to be installed. Some meant that blacklisting some drivers would do the trick. I tried it all.
By blacklisting the nouveau driver, and logging in to another tty, I installed the new Nvidia driver 331.38 64bit and then the GPU was recognized by BOINC. So I thought everything was OK now, but it wasn't. The GPU seemed to be running, but after 7minutes, the GPU temp. dropped to the same temp. as when it is idle! So maybe it runs for 7 minutes or so, using this setup, but I'm not sure at all. I made a test and a text file showing this. I will insert these data below.

After about 68 clean installation of Ubuntu Server 12.04.4, I decided to try a clean installation of Ubuntu Desktop 12.04.4 64bit, because X-server is installed and to try if this would make the GPU run. This is the result of the new installation:

From ubuntuforums.org and seti.berkeley.edu

I made several installation of both the Desktop and the Server, and the GPU can't be recognized before I blacklisted nouveau and others, stopped lightdm and installed the Nvidia driver 331.38. The difference in result between the 2 types of Ubuntu 12.04.4 is, that it runs continuously on Desktop but freezes of don't run at all on the Server edition. Did the same things on both Desktop and Server. Here's a text-file I made, testing the Server Setup by temp. of GPU and output from "boinccmd --get_tasks". What goes wrong with the Server Setup, is that it seems to run. You can see at http://asteroidsathome.net/boinc/hosts_user.php and the same for SETI, that the GPU has been recognized after using my "ToDo". But after some time, the GPU temp. drops! It drops to idle temp.! And the boinccmd output CPU time looks funny, until the temp. drops, then it looks normal for some time, then it freezes, I guess, and suddenly the CPU time starts to increase. The Estimated CPU time gets larger and larger. GPU cold as a ice cube ;(

Here's the text file/my test of the clean installed Ubuntu Server 12.04.4:

Code:

Code:
*****************************************************************************************************
AFTER CLEAN INSTALLATION OF UBUNTU SERVER 12.04.4 64BIT - NVIDIA DRIVER FROM MANUFACTURER - BOINC....
IT LOOKS LIKE IT'S RUNNING. LOOK AT THE CURRENT CPU-TIME, ESTIMATED CPU TIME, GPU TEMP. THE GPU TEMP. 
DROPS WHEN CURRENT CPU-TIME STARTS INCREASING!

# nvidia-smi -a |grep Gpu
        Gpu                         : N/A
        Gpu                         : 35 C

1) -----------
   name: ps_140207_16179_23_1
   WU name: ps_140207_16179_23
   project URL: http://asteroidsathome.net/boinc/
   report deadline: Thu Feb 27 23:31:38 2014
   ready to report: no
   got server ack: no
   final CPU time: 0.000000
   state: downloaded
   scheduler state: scheduled
   exit_status: 0
   signal: 0
   suspended via GUI: no
   active_task_state: EXECUTING
   app version num: 10111
   checkpoint CPU time: 0.698044
   current CPU time: 1.010883  <-- IS IT RUNNING? FUNNY TIME STAMP!
   fraction done: 0.004049
   swap size: 17324924928.000000
   working set size: 23929983.999999
   estimated CPU time remaining: 46193.601236  <-- LOOKS LIKE IT'S RUNNING OR SO IT SEEMS AT LEAST!

After 30 sek.:

# nvidia-smi -a |grep Gpu
        Gpu                         : N/A
        Gpu                         : 39 C

1) -----------
   name: ps_140207_16179_23_1
   WU name: ps_140207_16179_23
   project URL: http://asteroidsathome.net/boinc/
   report deadline: Thu Feb 27 23:31:38 2014
   ready to report: no
   got server ack: no
   final CPU time: 0.000000
   state: downloaded
   scheduler state: scheduled
   exit_status: 0
   signal: 0
   suspended via GUI: no
   active_task_state: EXECUTING
   app version num: 10111
   checkpoint CPU time: 1.159562
   current CPU time: 1.265546  <-- IS IT RUNNING? FUNNY TIME STAMP!
   fraction done: 0.007859
   swap size: 17325019136.000000
   working set size: 23998459.820312
   estimated CPU time remaining: 46051.810281  <-- LOOKS LIKE IT'S RUNNING OR SO IT SEEMS AT LEAST!

After 1 min.:

# nvidia-smi -a |grep Gpu
        Gpu                         : N/A
        Gpu                         : 40 C

1) -----------
   name: ps_140207_16179_23_1
   WU name: ps_140207_16179_23
   project URL: http://asteroidsathome.net/boinc/
   report deadline: Thu Feb 27 23:31:38 2014
   ready to report: no
   got server ack: no
   final CPU time: 0.000000
   state: downloaded
   scheduler state: scheduled
   exit_status: 0
   signal: 0
   suspended via GUI: no
   active_task_state: EXECUTING
   app version num: 10111
   checkpoint CPU time: 1.296961
   current CPU time: 1.395721  <-- IS IT RUNNING? FUNNY TIME STAMP!
   fraction done: 0.009764
   swap size: 17325019136.000000
   working set size: 23998463.967346
   estimated CPU time remaining: 45978.306833  <-- LOOKS LIKE IT'S RUNNING OR SO IT SEEMS AT LEAST!

After 2 min.:

# nvidia-smi -a |grep Gpu
        Gpu                         : N/A
        Gpu                         : 41 C

1) -----------
   name: ps_140207_16179_23_1
   WU name: ps_140207_16179_23
   project URL: http://asteroidsathome.net/boinc/
   report deadline: Thu Feb 27 23:31:38 2014
   ready to report: no
   got server ack: no
   final CPU time: 0.000000
   state: downloaded
   scheduler state: scheduled
   exit_status: 0
   signal: 0
   suspended via GUI: no
   active_task_state: EXECUTING
   app version num: 10111
   checkpoint CPU time: 1.431833
   current CPU time: 1.502592  <-- IS IT RUNNING? FUNNY TIME STAMP!
   fraction done: 0.011669
   swap size: 17325019136.000000
   working set size: 23998463.999490
   estimated CPU time remaining: 45916.704743  <-- LOOKS LIKE IT'S RUNNING OR SO IT SEEMS AT LEAST!

After 5min.:

# nvidia-smi -a |grep Gpu
        Gpu                         : N/A
        Gpu                         : 42 C

1) -----------
   name: ps_140207_16179_23_1
   WU name: ps_140207_16179_23
   project URL: http://asteroidsathome.net/boinc/
   report deadline: Thu Feb 27 23:31:38 2014
   ready to report: no
   got server ack: no
   final CPU time: 0.000000
   state: downloaded
   scheduler state: scheduled
   exit_status: 0
   signal: 0
   suspended via GUI: no
   active_task_state: EXECUTING
   app version num: 10111
   checkpoint CPU time: 1.705611
   current CPU time: 1.825205  <-- IS IT RUNNING? FUNNY TIME STAMP!
   fraction done: 0.015480
   swap size: 17325019136.000000
   working set size: 23998464.000000
   estimated CPU time remaining: 45735.682079  <-- LOOKS LIKE IT'S RUNNING OR SO IT SEEMS AT LEAST!

After 7 min.:

# nvidia-smi -a |grep Gpu
        Gpu                         : N/A
        Gpu                         : 40 C  <-- TEMP. DROPS FAST !!

1) -----------
   name: ps_140207_16179_23_1
   WU name: ps_140207_16179_23
   project URL: http://asteroidsathome.net/boinc/
   report deadline: Thu Feb 27 23:31:38 2014
   ready to report: no
   got server ack: no
   final CPU time: 0.000000
   state: downloaded
   scheduler state: scheduled
   exit_status: 0
   signal: 0
   suspended via GUI: no
   active_task_state: EXECUTING
   app version num: 10111
   checkpoint CPU time: 1.846315
   current CPU time: 24.267430  <-- AND NOW IT LOOKS LIKE NORMAL, COUNTING. BUT FIRST WHEN TEMP. DROPS ON GPU!?!?
   fraction done: 0.017385
   swap size: 17325019136.000000
   working set size: 23998464.000000
   estimated CPU time remaining: 45661.043715  <-- STILL LOOKS NORMAL, BUT THIS WILL STOP DECREASING AND START INCREAS LATER!?!?!?

After 9 min.:

# nvidia-smi -a |grep Gpu
        Gpu                         : N/A
        Gpu                         : 37 C  <-- TEMP. DROPS FAST !!

1) -----------
   name: ps_140207_16179_23_1
   WU name: ps_140207_16179_23
   project URL: http://asteroidsathome.net/boinc/
   report deadline: Thu Feb 27 23:31:38 2014
   ready to report: no
   got server ack: no
   final CPU time: 0.000000
   state: downloaded
   scheduler state: scheduled
   exit_status: 0
   signal: 0
   suspended via GUI: no
   active_task_state: EXECUTING
   app version num: 10111
   checkpoint CPU time: 1.846315
   current CPU time: 179.339600  <-- AND NOW IT LOOKS LIKE NORMAL, COUNTING. BUT FIRST WHEN TEMP. DROPS ON GPU!?!?
   fraction done: 0.017385
   swap size: 17325019136.000000
   working set size: 23998464.000000
   estimated CPU time remaining: 45508.722678  <-- STILL LOOKS NORMAL, BUT THIS WILL STOP DECREASING AND START INCREAS LATER!?!?!?

After 10 min.:

# nvidia-smi -a |grep Gpu
        Gpu                         : N/A
        Gpu                         : 35 C  <-- TEMP. DROPS FAST !!

1) -----------
   name: ps_140207_16179_23_1
   WU name: ps_140207_16179_23
   project URL: http://asteroidsathome.net/boinc/
   report deadline: Thu Feb 27 23:31:38 2014
   ready to report: no
   got server ack: no
   final CPU time: 0.000000
   state: downloaded
   scheduler state: scheduled
   exit_status: 0
   signal: 0
   suspended via GUI: no
   active_task_state: EXECUTING
   app version num: 10111
   checkpoint CPU time: 1.846315
   current CPU time: 332.599800  <-- AND NOW IT LOOKS LIKE NORMAL, COUNTING. BUT FIRST WHEN TEMP. DROPS ON GPU!?!?
   fraction done: 0.017385
   swap size: 17325019136.000000
   working set size: 23998464.000000
   estimated CPU time remaining: 45357.533616  <-- STILL LOOKS NORMAL, BUT THIS WILL STOP DECREASING AND START INCREAS LATER!?!?!?

1 hour later, approxmately:

# nvidia-smi -a |grep Gpu
        Gpu                         : N/A
        Gpu                         : 34 C  <-- SAME GPU TEMP. AS WHEN IDLE!

1) -----------
   name: ps_140207_16179_23_1
   WU name: ps_140207_16179_23
   project URL: http://asteroidsathome.net/boinc/
   report deadline: Thu Feb 27 23:31:38 2014
   ready to report: no
   got server ack: no
   final CPU time: 0.000000
   state: downloaded
   scheduler state: scheduled
   exit_status: 0
   signal: 0
   suspended via GUI: no
   active_task_state: EXECUTING
   app version num: 10111
   checkpoint CPU time: 1.846315
   current CPU time: 4815.600000
   fraction done: 0.017385
   swap size: 17325019136.000000
   working set size: 24051712.000000
   estimated CPU time remaining: 40951.029791  <-- AND NOW IT LOOKS LIKE NORMAL, BUT DOESN'T CRUNCH! GPU TEMP. SAME AS WHEN IDLE!?
Here's what I did when installing the Ubuntu Server. Same as I did with the Ubuntu Desktop:

Code:

Code:
Installation Todo:

0. Take sudo rights
# sudo su

1. Make sure "gcc" is installed by checking version "gcc --version". Install "gcc"
# gcc --version
# apt-get install gcc

2. Reboot
# reboot

3. Download newest/best Linux Nvidia driver!? e.g."NVIDIA-Linux-x86_64-331.38.run" from nvidia.com
# wget http://us.download.nvidia.com/XFree86/Linux-x86_64/331.38/NVIDIA-Linux-x86_64-331.38.run
# chmod 0755 *.run
or
# chmod + x NVIDIA-Linux-x86_64-331.38.run

4. Blacklist nouveau and other nvidia modules. Edit  "/etc/modprobe.d/blacklist.conf" and add "blacklist nouveau" etc. to the  bottom.

# vi /etc/modprobe.d/blacklist.conf

blacklist vga16fb
blacklist nouveau
blacklist rivafb
blacklist nvidiafb
blacklist rivat

5. Make needed to install Nvidia driver
# apt-get install make

5. Reboot
# reboot

7. Take sudo rights
# sudo su

8. Install the downloaded Nvidia driver
# ./NVIDIA-Linux-x86_64-331.38.run
Another problem by using the downloaded Nvidia driver 331.38, is that lm-sensors output looked like this before I installed the Nvidia driver. I need the GPU output for a shell script I made to control & watch CPU/GPU/HDD's:

Code:

Code:
# sensors
acpitz-virtual-0
Adapter: Virtual device
temp1:        +27.8°C  (crit = +106.0°C)
temp2:        +29.8°C  (crit = +106.0°C)

coretemp-isa-0000
Adapter: ISA adapter
Physical id 0:  +26.0°C  (high = +85.0°C, crit = +105.0°C)
Core 0:         +26.0°C  (high = +85.0°C, crit = +105.0°C)
Core 1:         +25.0°C  (high = +85.0°C, crit = +105.0°C)
Core 2:         +25.0°C  (high = +85.0°C, crit = +105.0°C)
Core 3:         +22.0°C  (high = +85.0°C, crit = +105.0°C)

nouveau-pci-0100  <-- THIS DISAPPEARS AFTER INSTALLATION OF NVIDIA 331.38 DOWNLOADED DRIVER!!
Adapter: PCI adapter
temp1:        +31.0°C  (high = +95.0°C, hyst =  +3.0°C)
                       (crit = +105.0°C, hyst =  +5.0°C)
                       (emerg = +135.0°C, hyst =  +5.0°C)
And as "oldfred" from UbuntuForums says "...If you install the .run from nVidia you are on you own as far as getting it to work correctly. With every kernel update you also have to recompile or actually create a new initramfs. If installed from repository it may not be quite as new from nVidia but will work with Ubuntu and dpkg will auto update on each new kernel.."

So, what we want, is to use a non-graphical environment like Ubuntu Server 12.04.4, No GUI what so ever. We control the boinc client with shell commands. All management is done by shell scripts and in command line. The servers are installed in RACK, and we never attach a monitor, only during installation. And only until it can be controlled by ssh.

Tex1954 is the most serious maker of a ToDo I've seen so far! And I've seen some! I make ToDo's for other people as well, but I'm not that good with Linux, yet
wink.gif
I only worked with Linux for a little more than a year.
I hope to hear from you in this forum, and then maybe, just maybe we can get this solved. There's a lot of people who would like to use non GUI computers for crunching data for ex. BOINC. To optimize performance and to be able to control a server and the boinc client by ssh.

These are the other discussions of the same issue:
http://ubuntuforums.org/showthread.php?t=2204590
http://setiathome.berkeley.edu/forum_thread.php?id=74108
http://setiathome.berkeley.edu/forum_thread.php?id=73032&postid=1478352#1478352

There's many more posts/threads all over the different forums, but these are the most serious. And those forums mostly related to the problem.

Please help us. Please help me o)

Kind Regards,
Dan Hansen
Denmark
 

·
Registered
Joined
·
201 Posts
Discussion Starter · #3 ·
Hello DarkRyder
wink.gif


Thank you, that's very kind of you
thumb.gif

We are so happy to hear from you in here, because we really need some help on this matter. My God, I haven't done anything else these last 3-4 weeks, than fighting this issue
wink.gif
We are really looking forward to hear from you again
wink.gif


Thanks again
wink.gif


Kind Regards,
Dan Hansen
Denmark
 

·
Premium Member
Joined
·
2,914 Posts
A few thoughts, I do have Ubuntu 12.04 (not server) installed and I am able to boot and run [email protected] and BOINC remotely with no attached peripherals using SSH and Putty.

I initially set up the system with a monitor attached and did install the desktop, I wonder if that somehow changes what gets installed?

I am using the 331.20 driver. The newest driver dropped [email protected] output by as much as 40% and introduced other problems that are widely reported if you google it.

There is a timing issue in the BOINC client that causes it to attempt to recognize the GPU before the driver has been loaded on boot. Restarting the client fixes this for me, but some people delay starting the client until the driver has initialised.

Good luck!
 

·
Registered
Joined
·
4,478 Posts
HI!

Welcome to the world of Linux!

I've spent years messing with this. Ubuntu as a whole doesn't much care about BOINC and BOINC related issues. Their primary concern seems to be pushing their Client and stupid shell... I've learned to hate Ubuntu.

After trying about every version of Linux out there, I discovered several that work well with BOINC. However, Linux morphs so often that one has to decided ahead of time what they want the Linux for. After installing it and getting it running well, disable all updates unless you know for sure you need them.

If you are like me, you want it to run BOINC only. No Libre office or other useless apps, just a BOINC cruncher. Perhaps you also use VNC like I do to talk to headless systems and want the GUI for that purpose. IF that is your situation, maybe I can help. I've found ONE setup that seems to work very well. But Ubuntu only ever worked for me well using 11.10 64Bit. All the later versions crapped out.

So far, I've never gotten AMD GPU's to run properly although I've knowledge of some others using AMD GPU's with some minor success. But, I've got Nvidia GPU's running fine.

On a side note, it seems some apps run better under windows vs. Linux on CPU tasks that I have to investigate as well. Still, the majority of my CPU farm with probably run Linux without GPU's.

There are several threads here that will tell you how to get BOINC running on various versions of Linux... my suggestion is toss Ubuntu out the door and go with LinuxMint Cinnamon.

http://www.overclock.net/t/1374845/6-months-of-pain-with-linux-now-i-got-it

biggrin.gif
 

·
Registered
Joined
·
201 Posts
Discussion Starter · #7 ·
Hi Tex,

Thanks for getting back to me
wink.gif


Well, it's a rack mounted system. A lot of 2unit high computers build into a rack. The only way I control them is by SSH/Putty. I use SSH/Putty from a windows 7 rig and a few ubuntu desktops/notebooks. But all controlling is in command line (CLI in your language?
smile.gif
I made some scripts that control CPU's/GPU/HDD temp., and these are shellscripts run by CRON. So it's all command line. Every single command I use to control Boinc is the the command line tool boincmd .
I've been testing the system for 2 weeks now, using the desktop edition. And even though the GPU are been used, the numbers are still less/smaller than when I ran the server edition. The numbers doesn't lie . So I have to find a solution. Many from berkaley.edu has ideas, some say "use slackware" some say, use windows.. well, I contacted the ubuntu community and let them know about all the problems many of us had regarding this matter. I know my problems means nothing to them. But maybe, when people from a different "flavour" solves the issue they would wish they had joined in.. Actually I find it very disappointing, the lack of interest.

Well, I hope we can solve the problem. As I've just wrote at hardwarecanucks, berkeley.edu test system arrives the day after tomorrow and then its "all hands on deck"

This is the hardware from test 1:

Project Headless RACK Linux Boinc Servers
Ubuntu Server 12.04.4 64bit
Intel i5-3470/4Gb Ram/Asus P8H61-MX
MSI GeForceGT610 2Gb
-
Nvidia v.nvidia-linux-x86_64-331.38
BOINC v.7.2.33 x86_64-pc-linux-gn

The next system will be:

Project Headless RACK Linux Boinc Servers
Ubuntu Server 12.04.4 64bit
Intel i5-3570K/8Gb Ram/Asus P8H77-M PRO
Asus GeForceGT640 2Gb PCIe 2.0 x16
Asus GeForceGT640 2Gb PCIe 3.0 x16

Then lets see what can be done about GPU and crunching on a headless linux server, even one from ubuntu.

My answer, may be a little "off".. If it is, Ill have to admit that I'm a little tired. It's 04:41 am. I've been fighting the problem of mapping a webdav share in windows 7. But I won that one
wink.gif


Again, thanks for your reply
wink.gif

Kind Regards,
Dan Hansen
Denmark
 

·
Registered
Joined
·
201 Posts
Discussion Starter · #8 ·
Quote:
bfromcolo
A few thoughts, I do have Ubuntu 12.04 (not server) installed and I am able to boot and run [email protected] and BOINC remotely with no attached peripherals using SSH and Putty.

I initially set up the system with a monitor attached and did install the desktop, I wonder if that somehow changes what gets installed?
Hi "Bfromcolo",

Well, I've 1 Ubuntu Desktop running right now. The problems went away, after the desktop edition of 12.04 was installed. But in the last 14 days, a test has revealed, that the crunching is not as good as when the server edition was installed and then only the CPU was doing the crunching.
I ordered a new system, a better CPU, a better GPU, Asus GT640. Actually it's 2 GT640, one PCIe2.0 and one PCIe3.0. I've found some sites containing the issue, and a guru from within the windows environment actually showed me one of the best guesses on how to solve the issue.
From berkeley.edu I learned that it's possible to make a headless cruncher using slackware. But I don't wan't to choose another flavour now, just because of a little problem.
It's just that I found this post http://www.overclock.net/t/1123532/guide-gui-linux-for-boinc-how-i-do-it-done-for-now and thought they might cracked the case
wink.gif


Well, all my computers (not notebooks and desktops of course) are Rack-Mounted, all in 2U cases which I imported from Germany. Here's a picture
Industrial 2U Cooler from Dynatron or JAC like this:


This is my Rack Mounted Linux Headless Boinc Crunchers. This is why I want it to be the Server Edition and all CLI:


This is the first test system which I'm running the Desktop Edition on. I'll show you the new system in a couple of days. Better CPU, 2 GPU and it will run CLI and Headless if its the last thing I do
wink.gif



The hardware I'm receiving tomorrow is:
Intel i5-3470K
Asus P8H77-M Pro
8Gb Kingston HyperX Genesis X2 Grey S.
Asus GT640-1GD5-L PCIe 2.0 x16
Asus GT640-1GD3-L PCIe 3.0 x16

Software & Drivers:
Ubuntu Server 12.04.4
Nvidia v.nvidia-linux-x86_64-331.38
LM-Sensors v.3.3.1 with libsensors version 3.3.1
BOINC v.7.2.33 x86_64-pc-linux-gn

I'll get back when I have better news
smile.gif


Kind Regards,
Dan Hansen
Denmark
 

·
Registered
Joined
·
201 Posts
Discussion Starter · #9 ·
Hi Guys,

It's time! The hardware is here and I will be putting (NOT PUTIN!) it together now. There's some issues when joining industrial hardware and commercial hardware.
Here's one. The bracket for the 2U cooler doesn't fit on several of Asus motherboards. Therefore I've been fitting the bracket so that it, well, fits
wink.gif
Look at this:




Here's the stuff for Headless Linux Boinc Server v.2.0.0:



The hardware I'm receiving tomorrow is:
Intel i5-3470K
Asus P8H77-M Pro
8Gb Kingston HyperX Genesis X2 Grey S.
Asus GT640-1GD5-L PCIe 2.0 x16
Asus GT640-1GD3-L PCIe 3.0 x16

If anybody knows how to install the parts of X which the GPU/CUDA uses to crunch, please don't hesitate to tell me. As you know, the Desktop Edition of Ubuntu does the trick. The problem is, that what we need is a headless cruncher where all controlling is done CLI
 

·
Registered
Joined
·
201 Posts
Discussion Starter · #10 ·
Hi,

OK, here we go. Just finished putting it together. The back row has been converted to Low Profile brackets so multiple graphic cards can be installed:




If anybody knows how to install the parts of X which the GPU/CUDA uses to crunch, please don't hesitate to tell me.

Kind regards
thumb.gif

Dan
 

·
Registered
Joined
·
201 Posts
Discussion Starter · #12 ·
Hello DarkRyder,

Thanks, thats nice of you to say
wink.gif


We still doesn't cracked the case! Tried several distros, in several different ways. OpenCL/CUDA, trough Wine, but the only one that works is when using GUI. I DONT WANT GUI! Not for a number crunching Rack-mounted rig
wink.gif


I just made a post at howtoforge, where I fell on my knee's to have some of the Linux Guru's to come to our aid. I've been trying for more than 1 1/2 month. Day in and day out.

I found a guy in the Netherlands, Mr. Gert-Jan, and he had some pretty exiting suggestions. And he did it using .deb files so that e.g. manual blacklisting of nouveau etc. wasn't necessary. But of course a new problem occurred. A warning was written a the Nvidia CUDA toolkit download site "...*** The CUDA 5.5 Debian packages are not compatible with Ubuntu 12.04 after the 12.04.4 LTS update. Please use the .run installer instead.." And being the big f...... idiot that I am, the .run file needs to many manually doings!

I also tried to make a installation of Ubuntu Server 12.04.4 and then installing the minimum required packages for X to get the GPU running. This, of course, didn't work either. Well it works, for 10 minutes it looks like. The GPU temp. rises and rises and then after about 10 minutes it drops. Not sure it did any work at all.

Well, I guess it may end in a compromise. A GUI rig doing a CLI rigs job.

After making all these attempts trying to solve the issue, I'm so confused I cant remember exactly what I tried and what I didn't try. I've been doing about 430-450 setups. 5-6 diff. distros. I hope, mayby someone will feel sorry for me and show me how it has to be done
wink.gif


I've been working now for 23 hours straight. Needs 1 hour of rest
wink.gif
I'll be back...

But thank you for the kind words... Nice to know someone keeps an interest in what you are doing, right
wink.gif


Primary discussions:
https://devtalk.nvidia.com/default/topic/418202/cuda-programming-and-performance/cuda-working-on-ubuntu-desktop-not-on-ubuntu-server/1/
http://setiathome.berkeley.edu/forum_thread.php?id=73032&postid=1484254#1484254
http://www.howtoforge.com/forums/showthread.php?p=310905#post310905

Primary info sites:
http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux/index.html
http://setiathome.berkeley.edu/cuda.php
http://boinc.berkeley.edu/wiki/GPU_computing
https://help.ubuntu.com/community/Cuda

Kind Regards,
Dan
 

·
Premium Member
Joined
·
5,054 Posts
I haven't gone through all of your issues/threads yet, but I will throw 12.04 LTS Server on one of my rigs and see what I can come up with. I have successfully used headless nodes for CUDA rendering, so I would think that BOINCing with a headless NVIDIA GPU should be possible.

Just to make sure that I understand what you would like to accomplish, can you confirm that the following is your goal?

  • Install BOINC on headless 12.04 LTS Server
  • Use NVIDIA 640 to crunch CUDA and/or OpenCL tasks
  • Control headless rig from Windows 7 using SSH/Putty

First off, I would recommend controlling BOINC with BOINCTasks from your Windows 7 rig, because BOINC Tasks has a very powerful GUI, that can do everything (and more), that you would want to do using boinccmd.

I will work on this over the next few days, and see what I can come up with.

Regardless of the outcome, your dedication to getting this up and running is pretty impressive.
:thumb:
 

·
Registered
Joined
·
201 Posts
Discussion Starter · #14 ·
Hi Guys,

I did it! Ubuntu Server Edition all CLI and no GUI, CPU & GPU running Boinc or is it the other way around?! Here's the proof:



Ooops... Here it is a little larger
wink.gif




I will make a nice ToDo when I'm all done
wink.gif
There's a couple of issues yet. I still need to check if it runs headless and with 2 or more GPU's. And then there's the shell script which checks temperature of CPU, GPU & HDD. And I would like to have some fan control as well. But the biggest issue is solved
wink.gif


I'll be back
wink.gif


Kind Regards,
Dan
 

·
Registered
Joined
·
201 Posts
Discussion Starter · #16 ·
Hi guys,

Sorry TicTac, I didn't notice your post before right now. Sorry!!! It was 5:30 am here in Denmark, and my eyes had a funny shape
wink.gif


DarkRyder, thank you for those kind words
rolleyes.gif


I'm so very interested in hearing what you find, because I want to make the fastest possible cruncher. I've just ordered the 3 system for testing a VM on a CLI environment like Ubuntu Server. A guy from berkeley had a great idea. So this will have to be tested too. But we did it!! We are now crunching without X-server running.

Next step. To check if it works with multiple GPU's and headless. And then I will install Aerocool hardware based Fan control. Newer mobo's like to control fans for you. This is not very good, when you are having GPU's and CPU running at 100%.
I've got a script where CPU/GPU/HDD temperature will be monitored and if something gets hot you are warned with an email, at the same time as the warning from X-vision, the fan-controller from Aerocool warns you. If something gets too hot (a limit chosen by you) the shell script run by CRON e.g. once a minute, will shut the server down. This way, if you should have a hardware defect there's 2 watchdogs keeping an eye on the system. I'm working on a little peace of electronic too. This will include the fans into the script. So that you will be warned if a fan crashes.
The 2U case I use, has 4 heavy duty fans, and the 5 and last fan which X-vision controls, will be the CPU fan of course. An industrial 2U fan which keeps the CPU at a nice 49 degrees celsius. Sensors will be placed around in the case, on the CPU of course, THE GPU's RAM, and PSU!

Lets see how it goes.

Thanks for the interest you guys, it's not everywhere you get that kind of attention
wink.gif


Kind Regards
Dan Hansen
 

·
Registered
Joined
·
201 Posts
Discussion Starter · #17 ·
Hi guys,

While doing these last things, I'll decided to order the 3 test system. This is a Asus board with 4 x PCIe 3.0 x16. Actually it has got 5, but the fifth can't be used for this. There's no room. This means that the 3 test will be a system with 4 GPU's. 4 Asus GeForce GT640 either PCIe 3.0 or 2.0. Doesn't matter in this case. It's not the bus speed I'm after, it's the GPU rendering time we like
wink.gif

And I will still be using the Intel i53570K CPU which has a great performance compared to the price and performance of i7-xxxx. Said in another way, it's not that much you gain using almost twice as much. In Denmark, one of the larger i7 cost about 500 US$. So should there be anyone out there who should be visiting Denmark in the near future, please don't hesitate to fill your pockets with i7's 6core for little me
wink.gif


This mobo looks like it hasn't got the issues from test system 1 and 2. On the flip side, there were several large disgusting component legs, with large solder points. Large as my a..
wink.gif
But with this mobo it looks like the bracket for the industrial 2U CPU fan is in the clear
wink.gif






The system will be fitted with 4 of these Low Profile Graphic Cards:



Here's a test where we can see that in windows environments the GT640 sucks! But when being put to the test on a 64bit Linux System something happens! This is why I'm using this card, and because it's the best low profile card. After all, I am trying to design a system which is not so pricey, but still does a very good job when running in a headless CLI environment
wink.gif




Kind Regards
Dan Hansen
 

·
Premium Member
Joined
·
2,914 Posts
Very interesting project you have going on, but you seem to have chosen relatively low end GPUs considering the performance (PPD) to cost ratio of higher end GPUs running in much less expensive systems. Are there particular projects you want to crunch where this architecture is beneficial?
 

·
Premium Member
Joined
·
6,454 Posts
Quote:
Originally Posted by DanHansenDK View Post

Hi,

And running multiple GPU's on the headless CLI based system (Ubuntu Server) also works now:

Test system 2:

# nvidia-smi -a |grep Gpu
Gpu : N/A
Gpu : 59 C
Gpu : N/A
Gpu : 44 C



Happy days
wink.gif
awesome man! good job, keep us up dated will all the good work man!
thumb.gif
 
1 - 20 of 347 Posts
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top