Here is a refresher for you (OP) and other people reading this thread:
Start overclocking on your stock voltage. Clock it until you can't go any higher without bumping the voltage. When you begin overclocking, you want to start with your CPU core speed and nothing else - ie; leave your CPU/NB and HT Link at or below 2000Mhz, and keeping your RAM down clocked to either stock speeds or equal to/less than 1333Mhz (666/667Mhz). You should overclock in nominal increments to discern stability, using either 5Mhz bumps, 10Mhz bumps, or slightly larger should you feel the need and/or are impatient.
If you don't want to spend the time doing it properly,
of course you can jump on ahead and set a processor from 3.2Ghz directly to 4Ghz and start stability testing. However, likely, you will not find stability.
Once you find your highest stable clock with whatever voltage works for your chip
up to 1.5v (from the 9xx to the 11xx series) or 1.55v (with proper cooling), then you can work on overclocking other parts of the system.
From here you can then work on your CPU/NB. To do this you should clock your RAM to AMD officially supported standard of 1333Mhz or lower (depends on what your RAM is rated for). You try to overclock the CPU/NB the same way you did with your CPU - you up the multiplier (which may report as a frequency instead of a multiplying number) with stock voltage until you find instability. Then you increase the voltage. You do this until you reach the highest stable clock speed on your CPU/NB without going over 1.5v (and provided you have proper cooling). The more speed and voltage you put on your CPU/NB the more heat your CPU will give off - remember that.
Next up you can overclock your RAM. You can do this two ways - through timings or through frequency. When you lower the timings of your RAM, say from a CAS latency of 9 to 7, you are essentially telling the RAM module to wait two cycles less for a refresh of information fed it from the CPU. You can also tighten the following timings which are related to the RAM chips themselves (this can get very technical if you get into sub-timings as well). This is why things load quicker, feel 'snappier' and benchmarks can improve when you do this (referred to as 'tightening timings'). You can also increase the frequency. If your RAM is rated for 1600, perhaps you can increase the frequency of the modules to 1700, 1800 or more. This will give you more performance in bandwidth, allowing for more data to be processed in the same amount of time as lesser clocked modules. You can increase your DRAM voltage but you should be careful, some RAM chips do not like loads of voltage and you have a large gamut of selection out there from 1.3v RAM to 1.9v+ RAM in the DDR3 range. If you have DDR2 then it will likely exceed 1.9v most of the time, dropping usually between 2~2.5v
Stability testing should be done separately. To use Prime 95 use this guide as a reference to isolate particular components for testing: Prime95: A Quick & Dirty Guide
(thanks to tmunn