Huge arrays of tiny, ultra cheap, slow ARM processors have been in the news for years but nothing has come out of it. I read that Microsoft had a research lab centered around this but nothing came of it.
Multi Processor servers are very focused on memory interfaces, latency for communication between chips, etc. I'm guessing that all of these important statistics would be very very poor with a huge array of weak chips, unless you're willing to implement expensive supercomputer interconnects (which obviously defeats the purpose). It just sounds like a pipe dream from PR people and reporters who don't understand what actually drives server performance.
That said, I'm sure there are some cases where this setup would work. Maybe search is one of these areas, but you can bet that if Google could save 90% of their power costs by using ARM, they would have spearheaded this process long ago. They are, after all, one of the first companies to use massive amounts of distributed consumer level x86 hardware for their datacenters.
edit: as for AMD making an actual ARM chip like NVIDIA did, I sort of doubt it simply because of the massive layoffs. Also, ARM chips are extremely low margin due to them being licensed designs. 90% of the work is in getting design wins, which is something that companies like Qualcomm, TI, etc are geared for (aka companies that have a vast array of commodity level chips), not companies like AMD and Intel. Making an ARM chip is much more like making an Audio DSP than making an x86 CPU. You have to sweet talk customers, and work with them every step of the way in the actual design of the chip, because the chip is designed for specific products (quite unlike x86). This is why NVIDIA has had such a tough time with Tegra, because they aren't good at this side of the business (which is actually the main job in a project like this). "Sorry, your chip uses .05 mw too much, or the package is too big, or too rectangular, or doesn't work with x,y,z" That's mobile chip design in a nutshell.
Edited by mechtech - 10/24/12 at 2:15pm