Are there many data centers using IB? Yes. SMP was common in HPC before it came down-market, likewise NUMA. Commodity processors have many features - vector instructions, specilative execution, SMT - first found in HPC. Power and cooling design at places liks Google and Facebook is heavily HPC-influenced as well. Certainly some things go the other way - e.g. Linux - but usually today's server design looks like last year's HPC design.

I'm not quite sure it's valid to write off SPARC as an architectural dead end when the current fastest computer in the world uses it, and the next crop of US competitors for that crown are all based on the Cell/BlueGene lineage. GPUs are also more broadly applicable than you might think. Besides video and audio processing, they can be used for many crypto-related tasks (witness their popularity for Bitcoin mining), various kinds of math relevant to data storage (e.g erasure codes or hashes for dedup), and so on. Many of their architectural features are also being copied by more general-purpose processors as core counts increase, as well.

Yes, high-end HPC is too obsessed with LINPACK. Nonetheless, it remains a good place to look when trying to predict the future of commodity servers. Even if ARM does displace x86 instead, many features besides the ISA are likely to come from HPC. Perhaps more relevantly, either outcome is still very bad for Intel.