I don't think physX is really turned off dude. I remember that selection being buggy back when I was testing that game a lot a few months back. I.E. sometimes the in game physX selector was 'wrong' ... it'd say physX was on high, but it really wasn't, and vice-versa.
These results you show/describe with the gpu and cpu usage match exactly with what I'd expect with physX running. The physX usage on your GPU doesn't show up when the card is also rendering, so that's why it looks like your stuck around 60% ... you can't 'see' the usage that's doing physX. Also, w/o a dedicated physX card, the cloth physX will run on the CPU, which causes very high CPU usage and some bottlenecking.
I don't remember what the solution was exactly, but try turning physX on High in the game, and setting physX to run on your GPU in the NVCP, and run the game again. Do you get pretty much the exact same results you do now? If so that's a sure sign that the problem is ... physX is running.