Check your drives and also make sure your caching enough of the video. Sometimes the defaults are too low. Also sometimes the default driver isn't the best one. If you can use dri then don't use the X11 codec. Use OpenGL or something else. Try all you have and see which one performs the best.
Which media player are you using?
Which media player are you using?