[Guide] Setting up multiple BOINC Instances - Overclock.net - An Overclocking Community

Forum Jump: 

[Guide] Setting up multiple BOINC Instances

Reply
 
Thread Tools
post #1 of 102 (permalink) Old 04-25-2017, 03:51 PM - Thread Starter
New to Overclock.net
 
mmonnin's Avatar
 
Join Date: Nov 2012
Posts: 5,826
Rep: 282 (Unique: 135)
[Guide] Setting up multiple BOINC Instances

The Pentathlon is coming up and I think one of the easier ways to bunker is to setup multiple instances of the BOINC on a single computer.

This has several benefits like:
  • Being able to easily shutoff network communication to a particular project while still allowing others to run.
  • Running multiple NCI Projects.
  • Getting more work for projects with a task count.
  • No VM overhead.
  • No switching HDDs or loosing production when one hard drive completes work.
  • No messing with hosts files.
Outlined below are the steps required to setup multiple BM instances in Windows. I am using Win7 so I'm guessing it would work in Win8/10. Scroll down for Linux version.

Setting up a 2nd BOINC Instance in Windows

1. Create a new folder for the project data

My BOINC folder is in C:\ProgramData\BOINC so I just made more folders there called BOINC2. Increment the # for each additional instance.

2. Edit cc_config.xml file

It is located in the main BOINC folder: C:\ProgramData\BOINC

Include the allow_multiple_clients option in cc_config.xml. Mine now looks like this:

PHP Code:
<cc_config>  
 <
options>   
  <
use_all_gpus>1</use_all_gpus>   
  <
allow_multiple_clients>1</allow_multiple_clients>  
 </
options>
</
cc_config
Also add in this line if the extra client is on a remote PC:

PHP Code:
<allow_remote_gui_rpc>1</allow_remote_gui_rpc
I also like to add in these lines to make task transfer faster:
PHP Code:
<max_file_xfers_per_project>10</max_file_xfers_per_project>
<
max_file_xfers>10</max_file_xfers
3. Create a batch file in the new BOINC folder

This will setup the folder and can be used to start the BM. Include these 2 lines inside the batch file. I name mine boinc2.bat and increment the # for each instance. (This will be a theme and generally a good practice.)

PHP Code:
"c:\Program Files\BOINC\boinc.exe" --allow_multiple_clients --redirectio --detach_console --gui_rpc_port 31418 --dir C:\ProgramData\BOINC2
exit 
Update the installation directory (if different) and the BOINC2 directory to wherever your BOINC folder is located and how it is named. BOINC2 if you're following along with me.

The default port is 31416 so increment the port # for each additional client. I have started at 31418 and went up from there.

4 Run the Batch file

It will start another BOINC instance pointing so the new data directory specified in the batch file. You should see another boinc.exe running as well. I end up with a folder and 12 files including the batch file when its done.

For more than one instance I created another batch file and updated the BOINC2 folder and port #. From there the 1st line of the batch file can be combined into one to start up all the instances at once.


Connecting BOINC Manager to the new instance. (Optional)

I say optional as from here on it can be controlled via BOINCTasks. Not everyone uses it so here's how to get another BM Instance running to connect to the new BOINC client.

1. Create new shortcut

I had a BM shortcut on my desktop already so I copied it and renamed it BOINC2 so I knew which instance it belonged to.

2. Copy password to the 2nd Boinc instance

One of the files created after running the batch file is gui_rpc_auth.cfg. Inside a bunch of random letters and numbers. Copy that.

3. Edit BOINC2 Shortcut

Edit the 'Target' line to include the new port # and new password. This is what mine looks like for my 2nd instance.

PHP Code:
"C:\Program Files\BOINC\boincmgr.exe" //n 127.0.0.1 /g 31418 /p 37cf632b718387f4675342371b4055cc 


I don't use passwords to connect to my clients, local or across the network. Your password will be different than mine.

If I remember correctly, the new BM didn't connect instantly the 1st time. It was searching for the host but it connected to the host after a few seconds. From here on it can be treated like another computer. Add/remove projects as you like. I always have to change the preferences to run while the computer is busy and run while CPU is in use.


Setting up BOINCTasks (Optional)

This is also optional. I use BOINCTasks to control all my BM instances locally and across the network. I never even have a BM instance open.

1. Add Computer

Select the Computer tab and then Computer in the File Menu and select Add Computer.



2. Give the computer a name

I incremented the name of the actually computer it was on, 3770k-2. That's not the complete computer name so it can be anything. Again I just incremented and kept the index consistent.

3. Specify the IP address

The instance is on the same computer as BoincTasks so its still localhost. Not sure if its needed but I've added the index at the end so BOINC2 has 'localhost2' as its IP address. It works for me.

4. Specify the Port

Use the same port # that was in the 1st batch file. BOINC2 for me was 31418.

5. Give BOINCTasks the password

Same thing here, paste in the password for BOINC2 from the gui_rpc_auth.cfg file.

After that I can control the Boinc instances with BoincTasks. I actually never have any BM windows open on my desktop let alone one for each instance. This is how some of my instances look like in BOINCTasks


I initially setup multiple clients to run Goofy for points and WUProp hours as it's an NCI app with badges but this should def help with the Pentathlon coming up.

Any app_config.xml or gpu.config files will have to be copied over to the new project folders created in the BOINC# folders.

I set this up awhile ago but just tried another instance but let me know if this works. Hopefully I didn't leave a setup step prior to all this.


Credit goes to @tictoc for adjusting the commands for Linux. I'm adding them into the OP.

Linux Version

I am running Mint so others running other distros or have installations in different directories may need to adjust the locations. Linux is case sensitive as a reminder. All the commands below can be entered in a terminal window which can be opened by pressing Control+Shift+T at once.

1. Create data directory for more BOINC instances

For every extra BOINC instances you want to run, just increment the # for each instance. I made mine right along the normal boinc-client folder with this command in terminal.

PHP Code:
sudo mkdir /var/lib/boinc2 
2. Create cc_config.xml file

Just like the Windows version, create the cc_config.xml and and paste these few lines in for easy access via remote BM or BOINCTasks access.

PHP Code:
sudo gedit /var/lib/boinc2/cc_config.xml 
Use your text editor of choice. I've used gedit and nano. Either works.

3. Paste text into cc_config.xml

Copy in these lines and save.

PHP Code:
<cc_config>        
 <
options>                
  <
use_all_gpus>1</use_all_gpus>                
  <
allow_multiple_clients>1</allow_multiple_clients>        
 </
options>
 </
cc_config
Also add in this line if the extra client is on a remote PC:

PHP Code:
<allow_remote_gui_rpc>1</allow_remote_gui_rpc
I also like to add in these lines to make task transfer faster:
PHP Code:
<max_file_xfers_per_project>10</max_file_xfers_per_project>
<
max_file_xfers>10</max_file_xfers
4. Create blank gui_rpc_auth.cfg

I've found it just easier to create a blank file in linux so create a blank file and save. It can probably be done the same in Windows but hey I already the wrote the windows version out.

PHP Code:
sudo gedit /var/lib/gui_rpc_auth.cfg 

5. Start BOINC Client Instance


Run this command to start up the BOINC instance with the adjusted port # and directory. tictoc was smart and made the last digit the same as the boinc folder number. 31422 and boinc2. Genius.

PHP Code:
sudo /usr/bin/boinc --daemon --allow_multiple_clients --gui_rpc_port 31422 --dir /var/lib/boinc2 
For more instances I adjusted the port # and boinc # folder. From here you can monitor with BOINCTasks or a BOINC Manager.

WUProp Setup

To allow for WUProp to gather data on any projects you're running on the additional BOINC instances, it will need an app_config.xml file to use the rpc_gui port that was setup with the client. Navigate to each of your boinc data folders and create an app_config.xml file and paste in these contents:

PHP Code:
<app_config>  
 <
app_version>   
  <
app_name>data_collect_v4</app_name>   
  <
plan_class>nci</plan_class>   
  <
avg_ncpus>0.01</avg_ncpus>   
  <
cmdline>-p 31418</cmdline>  
 </
app_version
</
app_config
If you've been setting up the same folder structure as me the folder will be located here:

PHP Code:
WindowsC:\ProgramData\BOINC2\projects\wuprop.boinc-af.org/app_config.xml
Linux
: /var/lib/boinc2/projects/wuprop.boinc-af.org/app_config.xml 
Adjust the port number on in the cmdline option to match that BOINC instance. Per my setup above my windows port for BOINC2 was 31418 and this is my file. Be sure to have BM re-read the app_config.xml files to pickup the change.


Edit: Add Linux version section.
Edit: Add WUProp setup.



Last edited by mmonnin; 04-25-2018 at 08:10 PM.
mmonnin is offline  
Sponsored Links
Advertisement
 
post #2 of 102 (permalink) Old 04-25-2017, 03:56 PM - Thread Starter
New to Overclock.net
 
mmonnin's Avatar
 
Join Date: Nov 2012
Posts: 5,826
Rep: 282 (Unique: 135)
I tried doing this in Linux as best I could. I was able to get another boinc executable to run and populate the BOINC2 folder but it still seems connected to the main client. When I run boinccmd --get_task info it shows the same thing even when specifying the port. BOINCTasks also doesn't connect either. It might have something to do with the exe runnning as user boinc in linux. Maybe someone that knows linux better than me, which isn't saying much, can get it to work. The basic contents of the batch file should work in linux.


mmonnin is offline  
post #3 of 102 (permalink) Old 04-25-2017, 04:17 PM
2+2=5
 
tictoc's Avatar
 
Join Date: Feb 2011
Posts: 4,474
I should be able to supplement this awesome guide with some Linux info when I get home tonight. thumb.gif


tictoc is offline  
Sponsored Links
Advertisement
 
post #4 of 102 (permalink) Old 04-25-2017, 04:35 PM - Thread Starter
New to Overclock.net
 
mmonnin's Avatar
 
Join Date: Nov 2012
Posts: 5,826
Rep: 282 (Unique: 135)
Quote:
Originally Posted by tictoc View Post

I should be able to supplement this awesome guide with some Linux info when I get home tonight. thumb.gif

Awesome, that'd be great.

I started out in the same direction as the windows guide but was getting errors trying to run this command. At 1st, it couldn't copy the gui_rpc_auth.cfg due to permissions. I had changed ownership to boinc cause linux does weird crap like that. I think adding sudo got it copied and the same files as windows were created.
Code:
sudo /usr/bin/boinc --allow_multiple_clients --redirectio-detach_console --gui_rpc_port 31418 --dir /var/lib/BOINC2

Trying to start another BM instance and Select computer (where I could enter another address) doesn't accept 127.0.0.1:31418 and the new password that was generated. I tried looking at the options for .desktop files but wasn't sure how to add the port and directory options like I did in Windows.

Edit: Starting up the client from terminal with the above command starts displaying the event log. Some of them mention not having and projects and one said this:
Code:
Data directory: /var/lib/BOINC2

So it seems part way there.


mmonnin is offline  
post #5 of 102 (permalink) Old 04-25-2017, 05:07 PM
New to Overclock.net
 
emoga's Avatar
 
Join Date: Mar 2014
Location: Edmonton, AB
Posts: 470
Rep: 29 (Unique: 14)
Nice guide thumb.gif@mmonnin



emoga is offline  
post #6 of 102 (permalink) Old 04-25-2017, 05:47 PM
Retired Staff
 
4thKor's Avatar
 
Join Date: Aug 2012
Location: Out there
Posts: 3,296
Rep: 35 (Unique: 27)
I'm going to attempt this on one rig. Probably sounds a lot more difficult than it actually is. My solution on the servers is to load Mint on several hard drives. This would be easier, I think.

<div class="post-sig post-sig-limit shazam usersig-click"><div class="reparse-sig-lineheight"><b><i>"Hell, there are no rules here, we're trying to get something done!" Thomas Edison</i></b><br><br><b><i>"You can't solve your problems with the same level of thinking that created them." Albert Einstein</i>


4thKor is offline  
post #7 of 102 (permalink) Old 04-25-2017, 06:21 PM
New to Overclock.net
 
Egilman's Avatar
 
Join Date: Dec 2015
Location: Tuvalu
Posts: 1,114
Rep: 34 (Unique: 18)
This is one heck of a neat idea!!!!! If we knew which projects were going to be picked we could start bunkering today....


Egilman is offline  
post #8 of 102 (permalink) Old 04-25-2017, 06:39 PM - Thread Starter
New to Overclock.net
 
mmonnin's Avatar
 
Join Date: Nov 2012
Posts: 5,826
Rep: 282 (Unique: 135)
Quote:
Originally Posted by emoga View Post

Nice guide thumb.gif@mmonnin

Thanks.

Quote:
Originally Posted by 4thKor View Post

I'm going to attempt this on one rig. Probably sounds a lot more difficult than it actually is. My solution on the servers is to load Mint on several hard drives. This would be easier, I think.

It really is just a couple of commands really. After I had figured it out I had seen a [H] admin write a script to do all the incrementing of folders, passwords (set is own so it was known), and ports to have it make like a 100 instances for Goofy. @fragamemnon had something for Goofy in Linux but I'm too much of a nincompoop for that.

Quote:
Originally Posted by Egilman View Post

This is one heck of a neat ideallllll If we knew which projects were going to be picked we could start bunkering today....

We could check for projects that have longer deadlines and take a gamble until the announcement. I'm guessing it could be one of the larger, more consistent projects for the long marathon. But who knows.


mmonnin is offline  
post #9 of 102 (permalink) Old 04-25-2017, 08:05 PM
2+2=5
 
tictoc's Avatar
 
Join Date: Feb 2011
Posts: 4,474
Linux Instructions

*The commands below assume that you installed BOINC via your distros package manager. I haven't had a chance to fire up a VM and test it if BOINC was installed via the script on the BOINC download page. I believe that BOINC is installed to the users home folder (~/BOINC) when the script is used, so you would just need to adjust the folder paths accordingly.

This is how I set it up for easy management, but you can change the directories to whatever you want. I would suggest placing your new data folder in your home folder, so that you don't have to mess around with permissions.

- Create a data folder for your new BOINC instance
Code:
mkdir ~/boinc2


- Create a new basic cc_config.xml (nano used in the example, but you can use whatever editor you like)
Code:
nano ~/boinc2/cc_config.xml
  • Code:
    <cc_config>
            <options>
                    <allow_remote_gui_rpc>1</allow_remote_gui_rpc>
                    <allow_multiple_clients>1</allow_multiple_clients>
            </options>
    </cc_config>
    


- Start the new instance
Code:
/usr/bin/boinc --daemon --allow_multiple_clients --gui_rpc_port 31422 --dir ~/boinc2/
  • The --daemon flag is not a true daemon. It is just BOINC's Linux equivalent to the --detach_console Windows flag
  • The daemon flag suppresses messages in the terminal, but it can be omitted, if you do not plan on using BOINC Manager to control the client, and would like to see the messages printed in the terminal
  • The gui_rpc_port can be anything above the default of 31416. I chose 31422 since it is my 2nd instance

- After the command is run, and your new data folder is populated with the standard files and the "notices" folder, shutdown BOINC.

- After BOINC is shutdown you can edit the created gui_rpc_auth.cfg file to an easier to remember password. Alternatively, you can delete the created password, and save the gui_rpc_auth.cfg file as an empty file to connect to the instance without a password.

To control your new client with BOINC Manager
  1. Start BOINC
  2. From a terminal start your newly created BOINC instance
    Code:
    /usr/bin/boinc --daemon --allow_multiple_clients --gui_rpc_port 31422 --dir ~/boinc2/
    
  3. Go to "File"->"Select computer..."
  4. For the hostname enter 127.0.0.1:31422 (or whatever port you specified in your command)

Now you can add projects, set your preferences, and all of the other things you do with your main BOINC instance. biggrin.gif


tictoc is offline  
post #10 of 102 (permalink) Old 04-26-2017, 01:41 AM
Not New to Overclock.net
 
fragamemnon's Avatar
 
Join Date: Oct 2010
Location: Bulgaria
Posts: 3,857
Rep: 249 (Unique: 187)
@tictoc, I've set up a couple of simple bash scripts which allow automated spawning of 240 BOINC instances (and optionally automatically attaching a project to each) for Linux.
If you want, I can provide the scripts. smile.gif

MEGATRON
(17 items)
GIGATRON
(10 items)
CPU
Ryzen 7 1700
Motherboard
Crosshair VI Hero
GPU
Gigabyte RX 580
RAM
2x8 G.Skill Trident 3200 CAS14 @ 3466 CAS14
Hard Drive
2x1TB WD Black RAID 0
Hard Drive
3TB WD Red
Hard Drive
Samsung 840 Evo 250GB
Power Supply
FSP Aurum PT 1200W
Cooling
Stock... hue hue
Case
Thermaltake Core X9
Monitor
Benq XL2411Z
Monitor
Dell U2412M
Monitor
Dell U2412M
Keyboard
KBC Poker II MX Clears
Mouse
Zowie ZA11
Mouse
old school Razer Goliathus
Audio
Aune T1 -> Yamaha A-720 -> Sennheiser HD650 | Sony MDR-XB500
CPU
Intel Xeon E5-2650 v4 @ 2.00GHz
CPU
Intel Xeon E5-2650 v4 @ 2.00GHz
Motherboard
SuperMicro X10DAL-i
GPU
nay
RAM
2x8GB (for now) Kingston DDR4 @ 2133MHz/C16
Hard Drive
Samsung 840 120GB
Power Supply
SeaSonic SS850-AM
Cooling
EK-Vardar F4 on an Arctic Freezer i11
Case
overrated
Operating System
Ubuntu Server 16.04
▲ hide details ▲


fragamemnon is online now  
Reply

Quick Reply
Message:
Options

Register Now

In order to be able to post messages on the Overclock.net - An Overclocking Community forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.
User Name:
If you do not want to register, fill this field only and the name will be used as user name for your post.
Password
Please enter a password for your user account. Note that passwords are case-sensitive.
Password:
Confirm Password:
Email Address
Please enter a valid email address for yourself.
Email Address:

Log-in



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off