Folding Rig Build Log - Overclock.net - An Overclocking Community
Forum Jump: 

Folding Rig Build Log

Reply
 
Thread Tools
post #1 of 15 (permalink) Old 06-12-2019, 04:49 PM - Thread Starter
I Can Change This???
 
tmontney's Avatar
 
Join Date: Aug 2012
Location: Wisconsin, US
Posts: 730
Rep: 13 (Unique: 12)
Folding Rig Build Log

Originally, I was just going to build something small to fold and be periodically active here again. But some things changed and I now take it as seriously as I did when I first started. I think those changes justify some sort of log. If you're interested, keep reading. (This guide may be useful to miners or other applications as well, who knows. The experience alone is invaluable to me, even if I stop doing [email protected])

Since I want to make things as efficient as possible, I'm building it like a mining rig. Although I'm quite versatile, I don't often build PCs (anymore). For these rigs, you have GPUs well past the norm. I'm only used to what you can properly fit in a case. The use of risers is new to me (although I have heard of them before). Much of my HW was purchased before my situation improved. Considering that, I am likely to make changes soon. For starters, let's list my hardware.

NZXT Gamma PC Case: https://www.newegg.com/black-nzxt-ga...82E16811146061
ASRock B360 Pro4 ATX Motherboard: https://www.asrock.com/MB/Intel/B360%20Pro4/index.asp
Intel Core i3-8100 CPU: https://www.newegg.com/core-i3-8th-g...-822-_-Product
Corsair 8 GB DDR4 Memory: https://www.corsair.com/eu/en/Catego...8GX4M1A2400C16
Seasonic Focus 750w Gold Power Supply: https://www.newegg.com/seasonic-focu...-201-_-Product
EVGA GTX 1070 SC: https://www.evga.com/products/produc...08g-p4-6173-kr

This was my original line-up. I was trying to go drive-less and do PXE booting. Although I still like the idea, it's obviously memory intensive, complicated, and (potentially) more expensive. Since it was taking days to figure it out, I just wanted to start folding so I put in an old laptop HDD I had lying around.





My situation changed recently when I investigated running the rigs at a datacenter. The price per kWh is cheaper. After some talks, I’ve started the process. Since I’ll be out of a rack, I figured I should build it like a mining rig. It's easy to add-on in the future and is very efficient (and well it mounts). I was initially concerned about the PCI-E slots dropping from x16, when enough slots are populated. It's my understanding now that bandwidth isn't important. Work units are only a few MB. In any case, they're virtually instantly loaded. It won't impact my PPD.

Hydra II 8 GPU 6U Server Mining Rig Case: https://www.amazon.com/Hydra-Server-...tronics&sr=1-2
Ubit PCI-E Riser Adapter: https://www.amazon.com/gp/product/B0...?ie=UTF8&psc=1
Apevia 120mm Fan Grill: https://www.amazon.com/gp/product/B0...?ie=UTF8&psc=1

First impressions were good, of the case. Seems very sturdy. Has just a toggle power button, comes with lots of screws, and even comes with 6 120mm fan grills.







First, the power supply. Had a bit of trouble getting the mount holes to line up at first. Then suddenly they did and it was smooth sailing from there.



Next, I got the motherboard in. No real issues here.



GPU and riser. This was a pain point for me. Since I've never done this, it was a bit unclear how to mount the GPU or power the GPU/riser. I figured out the GPU should sit in the slit slots. Before that, it was pretty unstable and I was worried it was going to fall out. That also made the riser underneath fit better. Second, I was confused about all the ports on the riser. Eventually I powered the riser with a molex connection and connected the 8-pin connector to the GPU.





The molex connector, not pictured here, is on the other side (too tight a space to see).



Finally, the hard drive. Easily mounted to the side of the case.



After my issue with powering the riser, everything fired up nicely. At first, I was wishing I hadn't ordered the extra set of fan grills (as I wasn't aware they were provided). Then I mounted one wrong, not knowing it had a flare to it. It broke one of the metal screw points off. Glad I ordered another set.



Going forward, I'm going to make some HW changes. All of the items below are on their way. First, the fans in place are temporary. The permanent ones are coming from China, so probably will arrive end of the month. Although high dBA, they have high CFM and will be in a DC (so no issue with noise).

Delta 120mm case fan: https://www.newegg.com/delta-afb1212...82E16835213001

Next, storage. I prefer a more streamlined solution and not to take up one of my PSU connectors for one hard drive. The motherboard has an m.2 slot, this will work perfectly.

Vaskey M.2 2280 60 GB Solid State Drive: https://www.newegg.com/vaseky-v900-6...0D9-00D6-00053

Lastly, fans again. My front fan almost didn't reach the board. Since I'll have 6 fans, it's definitely good to get a controller.

Deepcool FH-10 Fan Hub: https://www.newegg.com/vaseky-v900-6...0D9-00D6-00053

For software, the box currently runs stock Ubuntu 18.04. I've slimmed it down a bit (no GUI), but I'm definitely missing some things. I'll probably switch to Lubuntu, as it has a much smaller install disc. From there, removing anything not necessary (like the desktop).

6/13/19

Looks like everything should be ready for me to move in tomorrow.

6/17/19

I was too busy over the weekend, especially with Father's Day. Doesn't look like I'll be able to be up and running today. I hoping I could be up and running for this month's FAT.

6/19/19

I was able to check out the space. Internet still needs to be dropped in. My additional parts also arrived. I didn't think to check the dimensions on the fans. They're a lot deeper than regular ones I'm used to, so I can only fit the front 3. The ones by the GPU rails collide. Will need to find some slimmer, high CFM fans for alternatives.





As for the SSD, didn't have a screw handy. Perhaps it came with my used motherboard; otherwise, I'm on the hunt.



Once that's done, I'll be reinstalling the OS and trying to tidy up the cables. I also received the fan controller, but haven't placed it permanently yet. The fan cables are a bit shorter than my old fans. If I want to run them neatly, I'll need extensions. These look like they'll do nicely: https://www.amazon.com/gp/product/B0...?ie=UTF8&psc=1

6/25/19

Been on holiday. Last week, I got the networking information. Not sure if the physical cable has been dropped yet.

6/26/19

Heard today my Internet cable should be in tomorrow. Still not sure how to mount this thing, no word back from the MFG. For now, I have some shelves I can use. I also came across a bunch of i5/i7 laptops. Separately, I'm going to test how good they are. At best, I figure I can use them for FAT.

6/28/19

Talked it over again yesterday. Few things need to happen before I'm good to go. First, I'll be using shelves until I figure out how to rackmount the Hydra (company still hasn't responded). Second, the Internet cable still needs to be run (I believe it was being run yesterday). Third, everything 203v rather than 120v. I was gonna try using a spare old router but the adapter is up to 120v. Going to need to find an alternative adapter. Finally, power adapters. Their ends are different than the norm (probably standard for any DC). Maybe I'll be up by next week?

7/6/19

Decided to use my old sonicwall, as it had a 240v adapter. Got it in there but the shelf provided can't handle the weight. I'll be getting a different one soon.

7/12/19

On 7/8, everything was configured and installed. I've netted a little under 2.5m points since then, with both my CPU and GPU running. That's about 100k PPD off what I was expecting, probably loss due to the riser.

Since HFM.NET is Windows-only and I only have a raspberry pi (complicates due to ARM architecture), I still haven't gotten reporting working yet. For now, I can SSH FAH logs and extract PPD estimates. I am also working on my own HFM-style reporting system.

7/15/19

Been on work vacation (took off to work another job). Bought a 980ti from another member on here, hopefully should be here in a couple days. Still planning on swapping my mobo to improve lane bandwidth.

7/22/19

Bought a Sapphire Vega 56 from another member here. I'll be needing another PSU to handle 3 or more. The 980ti came in and should have it in this afternoon. (Thanks @snoball !)



After install, it's pulling about the same as the 1070. Pretty impressive. I also found the "load status indicator" a neat addition, but it only flashes green (indicating safe mode). Perhaps due to it being headless?

8/3/19

Been down for a couple days, due to a network issue. Hopefully should be resolved in the next day or two. To compensate, I've been folding on my Vega 56 at home. Eventually, the 56 will move to the DC when I get a better PSU. I've also been trying out different X16 riser cables. Both 25 and 30cm were too short. Got a 40cm on the way, hopefully that should work.

Looking to sell your GPU? I'm always looking to add more to my folding collection. PM me.



Last edited by tmontney; 08-03-2019 at 02:06 AM.
tmontney is offline  
Sponsored Links
Advertisement
 
post #2 of 15 (permalink) Old 06-12-2019, 04:49 PM - Thread Starter
I Can Change This???
 
tmontney's Avatar
 
Join Date: Aug 2012
Location: Wisconsin, US
Posts: 730
Rep: 13 (Unique: 12)
Thanks To

Data Holdings and Stack41 provide the space and data. Please check them out.

https://www.dataholdings.com/

https://www.stack41.com/


I am also working on a [email protected] stats project (like HFM.NET). It can be found here: https://www.overclock.net/forum/55-o...submitter.html

Looking to sell your GPU? I'm always looking to add more to my folding collection. PM me.



Last edited by tmontney; 07-24-2019 at 08:33 AM.
tmontney is offline  
post #3 of 15 (permalink) Old 06-12-2019, 07:34 PM
New to Overclock.net
 
mmonnin's Avatar
 
Join Date: Nov 2012
Posts: 5,846
Rep: 284 (Unique: 135)
Nice. With all that extra space, will you be using the other 4 PCI-E slots?


mmonnin is offline  
Sponsored Links
Advertisement
 
post #4 of 15 (permalink) Old 06-12-2019, 08:00 PM
Missing the Old days
 
Cerberus's Avatar
 
Join Date: Dec 2007
Location: Virginia Beach, Virginia
Posts: 4,754
Rep: 254 (Unique: 218)
notice any loss in PPD using a riser? I noticed almost a 30% decrease on my 980


"But I, being poor, have only my dreams. I have spread my dreams beneath your feet. Tread softly because you tread on my dreams."
Folding Rack
(9 items)
Work Folder
(7 items)
CPU
AMD r7 2700x
Motherboard
ASUS ROG Crosshair VI Hero
GPU
Nvidia Titan X Pascal
RAM
PNY Anarchy-X 2x8GB 3200Mhz
Hard Drive
256GB Plextor M8Pe m.2
Power Supply
Corsair RM750X
Case
big metal box
Operating System
Microsoft Windows 10 x64
Monitor
DELL S2716DG
Keyboard
Logitech G910
Mouse
Logitech G602
Mouse
Steelseries Mat
Audio
Logitech G602
Other
steam controller
CPU
AMD r7 1700x
Motherboard
MSI X470 Gaming Plus
GPU
ZOTAC GTX 980 AMP!
RAM
Geil EVO 1x4GB DDR4
Hard Drive
Toshiba 3TB
Power Supply
Silverstone ST-1000PT
Case
4U Rosewill
Operating System
Ubuntu 18.04LTS
Monitor
Headless
CPU
AMD FX8350
Motherboard
BIOStar TA970
GPU
AMD FURY X
GPU
Zotac GTX 960
RAM
Gskill 2x8GB DDR3
Hard Drive
Western Digital Blue 2TB
Power Supply
Corsair 650w
▲ hide details ▲


Cerberus is offline  
post #5 of 15 (permalink) Old 06-12-2019, 09:59 PM - Thread Starter
I Can Change This???
 
tmontney's Avatar
 
Join Date: Aug 2012
Location: Wisconsin, US
Posts: 730
Rep: 13 (Unique: 12)
Quote: Originally Posted by mmonnin View Post
Nice. With all that extra space, will you be using the other 4 PCI-E slots?
Eventually. I've been finding GTX 1070s that were sealed from a recent RMA, still with warranty, about half the price. Hopefully I can find more like that. I'll just keep adding on every 2 weeks to a month.

Quote: Originally Posted by Cerberus View Post
notice any loss in PPD using a riser? I noticed almost a 30% decrease on my 980
You noticed reduction with a riser? I did notice it started out at mid 750 then dropped about 50 to 75. I didn't give it much time, only ran 1 WU. Figured it was just the WU itself. I'll have to see once it's setup. Hopefully it doesn't, it doesn't affect mining. I don't know why [email protected] would be different.

Looking to sell your GPU? I'm always looking to add more to my folding collection. PM me.


tmontney is offline  
post #6 of 15 (permalink) Old 06-13-2019, 04:36 AM
New to Overclock.net
 
mmonnin's Avatar
 
Join Date: Nov 2012
Posts: 5,846
Rep: 284 (Unique: 135)
Quote: Originally Posted by tmontney View Post
Eventually. I've been finding GTX 1070s that were sealed from a recent RMA, still with warranty, about half the price. Hopefully I can find more like that. I'll just keep adding on every 2 weeks to a month.



You noticed reduction with a riser? I did notice it started out at mid 750 then dropped about 50 to 75. I didn't give it much time, only ran 1 WU. Figured it was just the WU itself. I'll have to see once it's setup. Hopefully it doesn't, it doesn't affect mining. I don't know why [email protected] would be different.
FAH uses the CPU during processing. Thats why there is performance loss with 1x riser cards. It's a small % each drop from like a 16x to 8x or 3.0 to 2.0.


mmonnin is offline  
post #7 of 15 (permalink) Old 06-13-2019, 08:24 AM - Thread Starter
I Can Change This???
 
tmontney's Avatar
 
Join Date: Aug 2012
Location: Wisconsin, US
Posts: 730
Rep: 13 (Unique: 12)
Quote: Originally Posted by mmonnin View Post
FAH uses the CPU during processing. Thats why there is performance loss with 1x riser cards. It's a small % each drop from like a 16x to 8x or 3.0 to 2.0.
I guess mining doesn't require any CPU overhead? I wonder why that is. I also wonder what the next best thing is if risers reduce performance.

Looking to sell your GPU? I'm always looking to add more to my folding collection. PM me.


tmontney is offline  
post #8 of 15 (permalink) Old 06-13-2019, 06:57 PM
Missing the Old days
 
Cerberus's Avatar
 
Join Date: Dec 2007
Location: Virginia Beach, Virginia
Posts: 4,754
Rep: 254 (Unique: 218)
Quote: Originally Posted by tmontney View Post
I guess mining doesn't require any CPU overhead? I wonder why that is. I also wonder what the next best thing is if risers reduce performance.
I was personally going to try x16 to x16 ribbon risers... Just get a motherboard with 4-5 pci-e x16 and do it that way?


"But I, being poor, have only my dreams. I have spread my dreams beneath your feet. Tread softly because you tread on my dreams."
Folding Rack
(9 items)
Work Folder
(7 items)
CPU
AMD r7 2700x
Motherboard
ASUS ROG Crosshair VI Hero
GPU
Nvidia Titan X Pascal
RAM
PNY Anarchy-X 2x8GB 3200Mhz
Hard Drive
256GB Plextor M8Pe m.2
Power Supply
Corsair RM750X
Case
big metal box
Operating System
Microsoft Windows 10 x64
Monitor
DELL S2716DG
Keyboard
Logitech G910
Mouse
Logitech G602
Mouse
Steelseries Mat
Audio
Logitech G602
Other
steam controller
CPU
AMD r7 1700x
Motherboard
MSI X470 Gaming Plus
GPU
ZOTAC GTX 980 AMP!
RAM
Geil EVO 1x4GB DDR4
Hard Drive
Toshiba 3TB
Power Supply
Silverstone ST-1000PT
Case
4U Rosewill
Operating System
Ubuntu 18.04LTS
Monitor
Headless
CPU
AMD FX8350
Motherboard
BIOStar TA970
GPU
AMD FURY X
GPU
Zotac GTX 960
RAM
Gskill 2x8GB DDR3
Hard Drive
Western Digital Blue 2TB
Power Supply
Corsair 650w
▲ hide details ▲


Cerberus is offline  
post #9 of 15 (permalink) Old 06-13-2019, 07:25 PM - Thread Starter
I Can Change This???
 
tmontney's Avatar
 
Join Date: Aug 2012
Location: Wisconsin, US
Posts: 730
Rep: 13 (Unique: 12)
Quote: Originally Posted by Cerberus View Post
I was personally going to try x16 to x16 ribbon risers... Just get a motherboard with 4-5 pci-e x16 and do it that way?
Because most boards I come across will downgrade to x8 or x4 if too many are populated. Do you know of any that don't do this?

Looking to sell your GPU? I'm always looking to add more to my folding collection. PM me.


tmontney is offline  
post #10 of 15 (permalink) Old 06-13-2019, 07:59 PM
New to Overclock.net
 
mmonnin's Avatar
 
Join Date: Nov 2012
Posts: 5,846
Rep: 284 (Unique: 135)
Quote: Originally Posted by tmontney View Post
Because most boards I come across will downgrade to x8 or x4 if too many are populated. Do you know of any that don't do this?
AFAIK all will do this as there are only so many PCI-E lanes. AMD offers the most PCI-E lanes for single CPUs with TR or go with 2P builds to get more full slots. Or take some slight drops in performance with 4x/8x lanes to use 1 PC.


mmonnin is offline  
Reply

Tags
fah , folding , log

Quick Reply
Message:
Options

Register Now

In order to be able to post messages on the Overclock.net - An Overclocking Community forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.
User Name:
If you do not want to register, fill this field only and the name will be used as user name for your post.
Password
Please enter a password for your user account. Note that passwords are case-sensitive.
Password:
Confirm Password:
Email Address
Please enter a valid email address for yourself.
Email Address:

Log-in



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off