I use gloves whenever I'm going to be physically removing a component that plugs into a socket on the motherboard. Also come in handy applying TIM if you like to spread it a little with your finger (as I do). Just slip off the gloves when you are done and voila all clean and tidy (also keeps oil and skin grease from contacting purified surfaces, dust, etc).
I wouldnt go out of the way to buy gloves strictly for pc building but having a wife who is a medic, we typically have boxes of them laying around so I figure why not.