Alrighty then, since you did not list your motherboard in your system specs, I won't be able to give too specific of advice since I won't know which brand of mobo you have or what BIOS you have.
Do you know how to get into your BIOS? When your computer is booting up you should see a message somewhere on the screen saying Press "_" to get into BIOS or maybe something else to that same effect, press whatever button it tells you to press, for my mobo it is DEL, once you are in your BIOS you can overclock your machine. let me warn you now, overclocking is a long and involved process which can easily take hours to find your best overclock! That being said it can also be very rewarding and even fun.
To overclock an Intel CPU you will basically want to raise your FSB (Front Side Bus) in 5MHz increments, raise it by 5MHz, save changes and reboot. If you were able to get into Windows, get back into your BIOS and repeat until you cannot boot into Windows.
Once you have reached the point where you cannot boot into Windows you have reached a "wall" to get past this wall you will have to do one of a number of things;
1) lower your FSB:RAM ratio, set your RAM to a lower speed (only do this if your memory is running above its rated speed)
2) raise your voltages, I would raise the CPU vCore first by 1 or 2 steps and try to boot, if it fails again try raising your motherboard's (chipset, NB etc.) voltage.
Is your CPU an 820 or 920? If its a 920 you should be able to overclock it very high, probably over 4GHz without any trouble, as you can see my 930 is at 4.21GHz with air cooling. If its an 820 you probably won't get much farther than 3.5GHz.
Also, we're gonna need more info about your computer http://www.cpuid.com/cpuz.php
download that program (CPUz 1.33) and run it, it will tell you all sorts of info about your computer, including which CPU you have (exact model) and which motherboard you have (mainboard tab) and the speed and brand of your memory, once you have found that info out, please fill out your system specs with as much detail as you can.