Get HWMonitor for temps and Prime95 for stability testing.
I would raise the FSB a bit each time to get a 100MHz gain on the processor, then run Prime95 for 10 minutes and check temperatures. Whenever it becomes unstable, raise the voltages a notch and try again.
It can be a tough balancing act between your memory and processor so I would turn down your memory clocks for now to remove them from the equation. This way, you will only need to worry about vcore and cpu-nb voltages, which you might need to notch either of them up to become stable. First try notch vcore, if that doesn't work, take off the vcore notch and notch cpu-nb, if that doesn't work notch vcore twice, rinse and repeat.
I wouldn't go above 1.45v on the vcore.
You want to keep cpu temps below 55C on your overclocks, when you get close to that temperature or reach your desired clock speeds, then you need to run Prime95 for 8 hours to confirm 100% stability.
If you are 100% stable, then raise your memory back up to its rated speed or OC it if you desire (may require raising cpu-nb and memory voltages), test again in Prime95 for 8 hours. If it stays 100% stable after that, you are good to go!
Don't go shooting for 4.0GHz right away. Overclocking is a lengthy process since every processor, motherboard, and memory reacts to overclocking differently than the next. If you take it one step at a time you will get the best overclock in the end... Keep in mind that overclocking is a bonus to performance, you might not be able to hit 4.0GHz!