First of all, have you got an OEM (prebuilt) machine? Overclocking capabilities are often, if not always limited/restricted by OEM boards, so you can't really get any more performance out of the hardware you have.
If you built the machine yourself or bought it custom, well that's a completely different story
Your BIOS should have options to change your base clock (FSB), CPU multiplier, and CPU voltage (VCORE). FSB x CPU multiplier is your final clock speed, so take your multiplier up before you have to take the FSB up. Once you've taken something up a little, boot into Windows (assuming it's stable) and run a stability tester such as Prime95 or Intel Burn Test for a few hours or so to confirm that it's stable with your settings.
Watch your temperatures while you're stress testing as well, you don't want them getting too high...
If you BSoD or get errors while running the test, then you'll have to increase the VCORE in the BIOS. This is where you need to be careful, raising it too much will generate excess heat. Please, please don't be one of those people who take it straight to something ridiculously high, lol. Increase the voltage in small amounts and test again, to see if it's stabilized. Keep going if you have too, just make sure your temperatures are fine.