Its your ram. D3 can utilize up to 512 megs as a frame buffer. Remember that it was a system killer about 3 and a half years ago so even todays lower end cards can struggle with it. I am not saying you have a lower end card but ya do...and I mean that in a nice way. I wouldnt think it would dip so low but I wouldnt rule it out either.
Its your Video ram and your proc man.
Something is definitely up, either due to the game not supporting the card or the current drivers having issues.