No no, I mean the theoretical analysis of neural networks. It's true that some authors attempt to do that, in fact even famous researches attempt to do this. However their arguments and analysis often breakdown in the general case and what their papers boil down to is that neural networks are good at modeling functions that they are good at modeling.

In fact the state of the field of machine learning is essentially the state of mathematics before Cauchy and Weierstrass.