It may be one of two things (sometimes it can be both):
1. Coil whine from the PSU;
2. Coil whine from the GPU;
V-sync solves both of the problems because it generally caps the load. If your card can render a game at 90fps but you only have a 60Hz monitor, then V-sync on will cap the fps at 60.
If your card is not being able to render a tough game at 60fps, V-sync can sometimes even go as far as capping it to 30fps, since that is the exact half of 60 fps, so the image stays in sync with the monitor's refresh rate. That, of course, also reduces the load on both the GPU and PSU.
You can try and open the case and try to hear where the noise is coming from. Some people think it's the GPU, and it turns out it's the PSU, and vice-versa.
By the way, did you buy a reference GTX570 ? If you did, it's likely you have exposed coils, instead of the more refined coils in more expensive, non-reference models. If that is the case, then it could be one of the coils making noise. More expensive models come with solid-state chokes (for example Gigabyte's Ultra Durable VGA line of cards) that can manage higher currents than traditional chokes, and they can also "help reduce the high frequency buzzing noises that occur when running under high electrical currents." - taken from Wikipedia
Coils vibrate under load, both on the GPU and PSU. A bit of special material is used to secure them in place so it does not vibrate, but sometimes it's not enough or is put in the wrong place. If you can track down where the noise is coming from, you can try using nail polish or a glue gun. Look it up on how to do it properly. Don't apply too much tough, because the coil needs to cool down. Coils are only held in place with the right amount of glue at the critical places and not all around, so they can cool down.
Edited by tpi2007 - 5/4/11 at 1:09am