Unless your room temperature is like, very hot, there's actually no reason to be this concerned about it because the heat being generated by your system is almost nothing. I mean, the 2500K is only at 4.3 GHz, and you have just one 5450 which itself produces almost no heat.
However, with that 500W Xigmatek, I'm not surprised you're hearing the fan. You could probably spend less money and get something like Corsair's CX430 and still be quiet. Still, a fanless PSU is pretty slick and you could have that with even a 400W fanless PSU.