Your platform is very old and will absolutely hold the 1080ti back...
I would go so far as to say you would benefit a lot from general system responsity by upgrading your CPU first with a cheaper GPU like the 1070 instead.
Adding more fans won't do much for your system, since the CPU will be a bottleneck no matter what. Getting a more powerful CPU will do much for your temps than more fans.
First get a new platform with a more powerful CPU and GPU, then think about adding two fans at the top blowing air out of the case. But until you remove the older components, it will not do much on its own.
For 4k gaming you do need a 1080ti to approach 60fps on max settings, but if you don't mind turning the settings down a peg the regular 1080 does the job as well.
As for processors, a 8600k from intel or ryzen 5 2600 both provide good results on 4k. Intel does do better overall due to higher clock however.
Another thing worth noting, I haven't seen any real performance gains from offloading tasks to a seperate GPU. It has had niche uses in the past, with hairworks and PhysX, but nowadays its better to keep it on one card to ensure stability. Troubleshooting SLI setups is a real pain.