The lack of a guiding theory has been something I've talked about with regard to big data and ML in general. This said, if you look closely at the math behind ML, it looks, not to surprisingly, like statistical mechanics.

So if we take some of the thought processes behind Stat Mech (or S&M as we used to call it in grad school), and you kinda squint your eyes hard to blur the less robust discussions you read about, you get the sense that ML is more about the "thermodynamics of information" than anything else.

I find this intriguing and definitely want to spend more time on this stuff.