To put it in perspective the default CPU voltage for the A10-6800K when it is running at its base frequency of 4100MHz is 1.3125V. There is a lot of spare capacity in there and you can stress test it up to around 4330MHz at the base voltage before it breaks down.
If you look at my table you'll see that to get 4600MHz I set CPU voltage to 1.35625V (7 increments above the base voltage) and loosen the VRM controls (CPU Loadline Calibration etc) to allow Prime95 to raise CPU voltage to a maximum of 1.432V (20 increments above base voltage) during the 30 minute run.
If you look at the default PState settings for the A10-6800K in this AmdMsrTweaker image you'll see three Turbo PStates for 4100MHz at 1.35V, 4200MHz at 1.4V and 4300MHz at 1.4375V (you can also see the Base State at P3), so you can see that 4600MHz at 1.432V is well within the range of what Richland can do.
In practice this is the most overclockable of all the APU processors and many people have taken it above 5000MHz with air cooling at voltages ranging between 1.5-1.6V. I wouldn't advise running it permanently at those levels and to watch the CPU temperature carefully while you are doing so, but anything you can run below 60C isn't going to do it any harm.
NB stands for North Bridge and denotes the part of the APU that deals with the functions that used to be dealt with by the motherboard's North Bridge chip before APUs were developed.
Unless you are overclocking the iGpu (the HD8670D Radeon part of the APU) from its default 844MHz you can leave NB Voltage at its default value.
NB Frequency should be at least 1.25:1 in relation to Memory Frequency where the given DDR3 (Double Data Rate) Ram speed is divided by two, so 2133MHz Ram requires 1.25 * 2133 / 2 = 1333MHz minimum NB Frequency and you will usually find the BIOS setting a higher default value than that. You can raise it above the default to increase the data flow but once you get above 1800MHz you may need to raise NB Voltage by 2 or 3 increments to support it.
As far as overclocking in BIOS and doing so in Windows is concerned the BIOS settings are read by Windows during Boot and those are the vales that your overclocking software will start from. Certainly you can use Overdrive and EasyTune to experiment and find an overclock but when you set it up in the BIOS you need to check what the operating values end up as in case there is a transaction during Boot.