First off I am going to say that you NEED to make sure that you have an after market cooling unit. That dinky little piece of crap that came with your processor just is not enough to keep your CPU cool. Something like the cooler master hyper 212 evo or this nzxt cooler would be a good entry cooler to keep your cpu happy http://www.newegg.com/Product/Product.aspx?Item=N82E16835146026
Second, you might want to think about changing that PSU. In fact you might want to change the PSU regardless of the situation. That thing has 2 12v rails and is NOT 80+ certified. That could turn into a system killer right there. I would strongly urge you to keep an eye out for a PSU sale and try to grab a corsair, seasonic, nzxt, or silverstone PSU. Cleaner power from the PSU usually means a more stable overclock.
Once you make sure you take care of those two issues, then you can continue on to overclocking.
In the BIOS there are going to be three main settings you want to look at: Multiplier, Baseclock, and voltage (the names of these vary depending on the mobo). So say for instance you had a multiplier of 34 and a base clock of 100 (this is your stock settings by the way). This gives you a clock speed of 3.4ghz. If you raised your multiplier to 40, then the speed would be 4ghz.
In your case you can only raise your multiplier by a couple of notches (I see some people seem to think the highest multiplier is 42 which would give you 4.2ghz with a 100 baseclock).
4.2ghz is sort of....boring to be honest. To get more horse power out of it you can also raise the baseclock. However, changing the baseclock also changes the ram speed. In this case from what I can tell from reading different guides on line it seems that the highest OC is a 42 multiplier and a 105 baseclock giving you a 4.4ghz OC which is much more respectable.
As for voltage, you really want to be careful and only use the bare minimum. The lower your volts, the cooler your CPU will run and the longer it will stay alive and happy. I would suggest no more than 1.35v for this OC, and realistically you could probably get away with less than 1.3v.
Once you have tuned your setting, it is time to boot up your PC. If it does not boot, simply resent the PC and raise your voltage. If your PC keeps rebooting on it's own, then I believe there should be a clear CMOS button on the back of your mobo. Simply press that when the PC is OFF and it will reset your BIOS settings. If your PC boots, then download CPU-Z, prime 95, HWmonitor, and intel burn test. Open CPU-Z, HWmonitor, and prime 95. Keep an eye on the cpu temps by use of the hwmonitor, and run a "blend" stress test with prime 95. Make sure that your temps are not getting too hot (Above 80C I start to worry a little....above 90C I worry a lot....above 95C I shut the PC down and try to lower the voltage or get a better CPU cooler).
I let prime 95 run over night to make sure it is stable. In the morning I run intel burn test with max amount of ram and a bunch of passes to make sure that the ram is stable as well as the CPU.
If you PC passes all the tests, then you should be fine. If it doesn't pass a test, raise the voltage and start allllllll over again