I think it is basic economics supply and demand for the most part. There is also imperfect information and the need for good negotiation.
Look at collectible cars or houses high demand areas. People are going to pay above asking price.
I think the same is true of very talented individuals. I was seeing compensation figures of 1M+ for C++ developers that can design high speed trading systems last year. I think AI researchers are able command higher compensation due to the lack of supply.