The only real problem right now is getting the mineral oil, it comes to about $100 with shipping. We are going with a tank that is bigger than the Eclipse tank, at 5.5 gallons. Hopefully with the bigger width and depth we could get the motherboard down about 3-6 inches from the bottom to maximize cooling. We have been researching for about 10 hours when we got this crazy idea and it certainly seems viable. If we don't use this method, Matt would most likely get a case for $100 so I consider the extra for the oil well deserved. Motherboard mounting will be somewhat of an issue if we cannot lay it down. I have come up with the following: put a tray in the bottom, mount there, mount to the top of lid/side and have the motherboard hang down into oil, overhangs on both sides and mount mobo on one, have adhesive above oil and hang motherboard.
After researching a very good amount we see that a pump would greatly aid in circulating oil and speeding up the convection process. We are considering experimenting with radiators and other various forms of cooling including homemade heatsinks. We are fairly confident in our abilities to overcome any problems we run into and look forward to a successful project. Hopefully the parts will be gathered (I am upgrading my system and giving him a few parts) and ordered by next weekend and we can update this thread with some pics and results.
Total cost will come to ~$150