So I got another CPU fan intel sent me and I still had the same issue. So I decided to get the Coolermaster 212 Evo and the noise is gone. The fan turns right when I boot and it stays quiet the whole time. I am getting 33 degrees on normal use and on load with prime 95 I'm getting 55 degrees max. When playing games I'm getting up to 45 degrees.
The load temps seem good to me, can the idle be lower or that's normal?