Microsoft is in process of building optical links between existing datacenters to create meta-clusters, and I'd expect that others like Amazon and Meta may be doing the same.
Of course for Musk this is an irrational ego-driven pursuit, so he can throw as much money at it as he has available, but trying to sell AI when you're paying 10x the competition for FLOPs seems problematic, even you you are capable of building a competitive product.
In terms of who's got a lead on cheap (non-NVIDIA) hardware, I guess you have to give it to Google who are on their 6th generation TPU.
https://jack-clark.net/2025/02/17/import-ai-400-distillation...
https://centreforaileadership.org/resources/deepseeks_narrat...
If you’re using your compute capacity at 1.25% efficiency, you are not going to win because your iteration time is just going to be too long to stay competitive.
xAI bought hardware off the open market. Their compute edge could dissappear in a month if Google or Amazon wanted to raise their compute by a whole xAI