If you have an AMD card you can use, there's a ridiculously good debug OSD built into the open source drivers. It can track all kinds of stuff and it can draw graphs. Using it looks like this:
Code:
GALLIUM_HUD=fps,GPU-load,cpu0+cpu1+cpu2+cpu3 glxgears
This will show three graphs, first one for fps, second one for GPU usage, and the third one CPU usage for four cores.
The sources for the stuff in the OSD don't have to be from the graphics driver, the OSD can also show stuff from elsewhere in the system. You can make it show CPU temperatures for example, and it can also show somewhat obscure stuff like network traffic and disk reads/writes. You can get a help text by running "GALLIUM_HUD=help glxgears".