To improve audio quality with pulseaudio present: Everything in a code box is done in a terminal

1) Make a backup of default settings:
[CODE]sudo cp /etc/pulse/daemon.conf /etc/pulse/daemon.conf.old
2) edit the daemon.conf to use the best resampler and the highest quality sample format:
[CODE]sudo nano /etc/pulse/daemon.conf
3) find the following:
resample-method =
default-sample-format =

4) change both to: (make sure to remove the ";" before default-sample-format)
[CODE]resample-method = src-sinc-best-quality
default-sample-format = [S]float32ne[/S]
Here you should put the results from running pacmd list-sinks and the information found on the line called sample spec. The first piece of information should be the sample format your card will work best with. For my case the output was
sample spec: s32le 2ch 44100Hz
thus my sample format is s32le. The float32ne is simply a generic method.

SECOND ISSUE: If you know that you will be using audio of a higher frequency (AKA something above 48KHz such as a Bluray HD audio track). Then you will likely want to change this area.
[CODE]alternate-sample-rate = 48000
to this
[CODE]alternate-sample-rate = 96000
This will prevent resampling of audio that is @96KHz but it will create resampling of any audio @48KHz. As of right now I have no idea if more than 2 default sample frequencies/rates can be used. So you're stuck with the choice of of which you want to avoid resampling. In my experience most sources even Bluray tracks seem to top out at 48kHz so I'd say leave it at 48000.

change this accordingly (make sure to remove the ";" before each setting)
3 for 2.1 aka 2 speakers and a sub and (While 3 for 2.1 is valid and supported most 2.1 speakers don't work properly with such a setting so you should tinker with that to figure out what works best for you)so 6 would equal 5.1 and 8 would equal 7.1 (you'll have to add the locational name of each speaker you have in the channel map to make it work properly)
(a generic 5.1 setup edit to your needs)
[CODE]default-sample-channels = 6
default-channel-map = front-left,front-right,rear-left,rear-right,center,subwoofer
SIDE NOTE: You might want to experiment with swapping subwoofer with lfe to see which one sounds best for you and your specific sound hardware. But make sure your hardware can handle the lfe as it is noted on occasion the lfe can destroy speakers or subwoofers.

Lower these numbers as much as you can (at 0 pulse will no longer work and ALSA will be used instead) but be careful they may introduce problems and flash will likely become a problem or may not work at all.
Apparently I goofed up here...though this is purely subjective to each person; the recommended thing to do is set a higher fragment number and size. For me I personally have had better results (lower CPU usage) with as few fragments as pulse will allow me (in my case it was 2) and as large a fragment size as I felt necessary (I'm at 400 right now). The changes here are really up to you. This section has been a source of problems as there's much misinformation out on the net. The safest bet outside of default is to use more fragments and a fragment size of 50 or greater. This however is a generic method. If you want to know how to figure out the specific for your configuration setup for this part of audio you'll need to be ready to do some math and have a read from the steps laid out by these guys at the mint forum:
One note is that you should use your info from running pacmd list-sinks and then follow steps 2-3 in the mint guide.
[CODE]default-fragments = 1 (4 is default)
default-fragment-size-msec = 1 (25 is default)
Find "nice-level" and change it from the default -11 to a high negative (remember -20 is max and not recommended). Remember as usual to remove the ; before "nice-level".
[CODE]nice-level = -15 #this setting increases the frequency/share of CPU time that Pulseaudio has access to
SIDE NOTE: Not confirmed by anyone else yet but it seems changing the priority has a negative effect on KDE's system sounds so they'll cause a second or two of glitching/blocky/ugly sound if something else is using audio. No idea why but the priority change might negatively effect sounds from DEs such as Gnome, KDE. Doesn't seem to cause problems with other applications using sound though. UPDATE: Skype is another thing that doesn't like the nice level or latency changes. If you make changes similar to this then make sure not have anything trying to use the sound server at the same time as Skype as it's again one program that just doesn't play nice with others. It works perfectly fine if nothing else tries to use the sound card so don't worry there.

Warning: Spoiler! (Click to show)
Make the fragment section of the config look like this:
[CODE]default-fragments = 0
default-fragment-size-msec = 0
It should be noted that this will negate any of the changes made prior to this point as this is a lazy way of getting rid of pulseaudio without having to actually uninstall pulse audio. It works by simply causing pulse not to load as this is an incorrect setting. Thus, pulse should no longer show up in your process list or return a follow from running
[CODE]ps -A | grep "pulse"
. This means that ALSA will be in charge of everything and configuring it is a whole separate topic. However, it seems most people believe that one does not have to configure ALSA at all as it should "just work" by default so you shouldn't need to do any config work with it.

An alternative and likely better way to disable Pulseaudio is to change the following in the /etc/pulse/client.conf file:
[CODE]; autospawn = yes
[CODE]autospawn = no
7) exit and save changes

8) Next you need to kill pulse so the old config settings are no longer in effect
[CODE]pulseaudio --kill
9) Then you should check that your config is valid
[CODE]pulseaudio --check
10) If it returns nothing then your config is fine and you can restart pulse with your changes implemented
[CODE]pulseaudio --start
NOTE: These changes may increase CPU load and the sample format may not be supported by the audio hardware on your mobo or card. Lower the number from 32 to 24 to 16 until it works. Otherwise use the original default setting. It should be noted that using a float setting will always result in lower CPU load and better sound quality.

Example of how it should look. Remember any changes made need to have the # or ; before each line removed in order to take effect. This file has real time enabled which requires changes to /etc/security/limits.conf before working; an example of that file will be shown as well.
Example daemon.conf (Click to show)
[CODE]# This file is part of PulseAudio.
# PulseAudio is free software; you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
# PulseAudio is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# General Public License for more details.
# You should have received a copy of the GNU Lesser General Public License
# along with PulseAudio; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
# USA.

## Configuration file for the PulseAudio daemon. See pulse-daemon.conf(5) for
## more information. Default values are commented out.  Use either ; or # for
## commenting.

; daemonize = no
; fail = yes
; allow-module-loading = yes
; allow-exit = yes
; use-pid-file = yes
; system-instance = no
; local-server-type = user
; enable-shm = yes
; shm-size-bytes = 0 # setting this 0 will use the system-default, usually 64 MiB
; lock-memory = no
; cpu-limit = no

; high-priority = yes
; nice-level = -11

#This is the realtime setting section. The ";" are gone meaning these are active
 realtime-scheduling = yes
 realtime-priority = 49 #9 is the recommended setting use a higher number at your own risk 49 is my setting.

; exit-idle-time = 20
; scache-idle-time = 20

; dl-search-path = (depends on architecture)

; load-default-script-file = yes
; default-script-file = /etc/pulse/

; log-target = auto
; log-level = notice
; log-meta = no
; log-time = no
; log-backtrace = 0

resample-method = src-sinc-best-quality #this is the best resampler there is.
#If your CPU is weaker see secret rabbit code or stick to speex-float-10
; enable-remixing = yes
; enable-lfe-remixing = no

flat-volumes = no #this disables system wide EQ IIRC which you want for fidelity sake 

; rlimit-fsize = -1
; rlimit-data = -1
; rlimit-stack = -1
; rlimit-core = -1
; rlimit-as = -1
; rlimit-rss = -1
; rlimit-nproc = -1
; rlimit-nofile = 256
; rlimit-memlock = -1
; rlimit-locks = -1
; rlimit-sigpending = -1
; rlimit-msgqueue = -1
; rlimit-nice = 31
 rlimit-rtprio = 49 #this must match the realtime-priority you set above will also need to edit the /etc/security/limits.conf file to allow realtime. See that file example.
; rlimit-rttime = 1000000

; default-sample-format = s16le #s32le, float-32 are other available methods. AFAIK s24le implies s32le.
; default-sample-rate = 44100
; alternate-sample-rate = 48000 #change this the highest kHZ rate of anything you own IE 96000 if a true bluray.
; default-sample-channels = 2 #configure this to match the number of speakers you have...
#...6 equals a 5.1 system. 8 equals a 7.1 system.
; default-channel-map = front-left,front-right

default-fragments = 25 #see the guide for a link on figuring this out. Best to leave it at 5 or increase it if... notice dropped or clipped playback
default-fragment-size-msec = 364 #see the guide for a link on figuring this out. Best to leave it at 25 or increase... if you notice dropped or clipped playback

; enable-deferred-volume = yes
deferred-volume-safety-margin-usec = 1
; deferred-volume-extra-delay-usec = 0
Example /etc/security/limits.conf. For this change the rez name to match your user name when you add those two lines.
Limits.conf example (Click to show)
[CODE]# /etc/security/limits.conf
#Each line describes a limit for a user in the form:
#<domain>        <type>  <item>  <value>
#<domain> can be:
#        - an user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#        - NOTE: group and wildcard limits are not applied to root.
#          To apply a limit to the root user, <domain> must be
#          the literal username root.
#<type> can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#<item> can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open files
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit (KB)
#        - maxlogins - max number of logins for this user
#        - maxsyslogins - max number of logins on the system
#        - priority - the priority to run user process with
#        - locks - max number of file locks the user can hold
#        - sigpending - max number of pending signals
#        - msgqueue - max memory used by POSIX message queues (bytes)
#        - nice - max nice priority allowed to raise to values: [-20, 19]
#        - rtprio - max realtime priority
#        - chroot - change root to directory (Debian-specific)
#<domain>      <type>  <item>         <value>

#*               soft    core            0
#root            hard    core            100000
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#ftp             -       chroot          /ftp
#@student        -       maxlogins       4
rez              -       nice            -18 #this must match or exceed what is in your daemon.conf so it must be >= to nice-level
rez              -       rtprio          49 #this must match what is in you daemon.conf file so if you use 9 this must be 9

# End of file
A possibly helpful suggestion for laptop users with either NVIDIA and/or ALC889 onboard soundcard
Added October 30, 2014

From @Evil Penguin
Welp, looks like I finally got the audio performing the way it should.
It literally took me several hours of trial/error and research to get these pesky audio issues sorted.

The ALC889 can't have a default sample rate of 44100 if the sample format is higher than s16le (doing so will result in a huge cluster of issues).
s16le and s32le appear to be the only sample formats that work. src-sinc-best-quality resampler appears to run into a CPU bottleneck and causes distortion so I changed it to the next best thing (src-sinc-medium-quality) and it works without issue. I have NVIDIA HDMI audio and when it's on, it has issues with the modified PA settings and crashes/crackles so I turned it off via pavucontrol (configuration tab).

I've settled with:

resample-method = src-sinc-medium-quality
default-sample-format = s32le
default-sample-rate = 48000
alternate-sample-rate = 96000

I now have a better understanding of why there's popping noises when messing around with the desktop environment while listening to music.

Taken from the Gentoo wiki:
No guarantees on actual latencies

Currently PA provides whatever latency at that moment is possible be it some milliseconds to hundreds of milliseconds without regard to what applications ask for.

In case of buffer under-run latencies are never decreased
Currently, if a buffer under-run occurs, PA buffers for longer increasing latency, but it then never tries to buffer for less until restart.
If the buffer isn't large enough, you'll likely hear audio distortion (in my case crackling).
PulseAudio automatically increases the latency/buffer size every time there's a buffer under-run.
Alternatively you can set the latency yourself by disabling timer-based scheduling but I wouldn't advise that unless you know what you're doing.