The debate seems to be between trusting your compiler to generate good code or writing close-to-metal code yourself.

There might be a third option though - code generation in a HLL. Coconut is a nice example of this:

http://www.youtube.com/watch?gl=GB&hl=en-GB&v=yHd0u6...

Instead the compiler being a black box, Coconut is structured as a set of libraries for code generation, analysis and optimisation. They report outperforming c SIMD by up to 4x on the cell architecture.