Main obstacles in OCing are temperatures and voltages.
Donwload HWMonitor here
to monitor your temperatures and VCore.
You are watching for temperatures verging on and reaching 60 degrees. Although you "can" go higher than this I would not recommend it. Overclocking with stock cooler is not recommended but is doable for small overclocks.
Secondly you are going to need to increase your VCore in the BIOS as you increase the frequency of the processor. Use Prime 95
to stress your processor and use HWMonitor at the same time to monitor the temperatures. As you increase the frequency of the processor you will need to run Prime 95 multiple times after every boot, if it passes then the voltage is ok, if it fails then the voltage is too low and needs to be increased by a few milivolts until it is stable.
Thirdly as you increase your processor speed you will also be increasing the speed of your RAM, whilst overclocking I would recommend setting the RAM multiplier to its lowest setting because if the RAM runs too fast your computer won't boot and this will interfere with getting your processor stable.