Try Eigen then, where people were tweaking every last ounce of performance. Even then, it has problems matching MKL or nVidia libs for ultimate performance sometimes.