Sometimes it's not the BIOS but the timing straps in the chipset that determine what speed the memory bus boots to. In the T3400 chipset if you force the hidden 400 FSB DDR3 runs at 1600 or 1066 but not the normal 1333. DDR2 will only boot at 800, not the 1066 allowed by the 333 FSB. The locked Dell BIOS may impose further limits. With modern GPUs that have 2GB DDR5 memory, memory speed isn't the issue it once was. I would gp for all the CPU clock you can get, and then the max GPU you can get to work. ITX can be made to fit.
My advice for the Optiplex 755 would be Q6700 (they're all G0 stepping) with a pinmod to 333FSB ,and VID pinmod to 1.43V., either D9729 heatpipe cooler (bolts in), or the bettter T9303 which requires moving the holes in the cover to match. An expansion slot blower will clear the heat out of the expansion slot area. PSU options are aftermarket, or T3400 375W, or 525W will fit also. Cable lengths will be a better match with those. Sorry for the late reply I've been offline for a couple weeks. Basicaly you're stuck with DDR2 800 which is just as fast as DDR3 1066. An extra $50 in the GPU will do a lot more for you than memory clocks.