1. A way to interpret math. E.g. given a computation you might interpret some values as probabilities. 2. A particular set of methods which people use to analyze information as well results of such analysis.

The problem with "just statistics" is that 99% of people would understand it as #2. But deep learning is very much not like "normal" statistics.