> Any automatically trained net with more than a few dozen neurons is virtually impossible to analyze and understand.
While it is still an open area of research to try to get neural networks to explain themselves, the solution to this specific case is quite simple (and has been used in practice, I think):
You flip random bits/pixels of the input image and get the neural network to tell you whether the image looks less or more like a tank. Continue this process in a self-supervised loop and eventually you reach what the neural network thinks is a "maximally tank depicting" image. In this case what you would see is a picture of grey skies and no tanks, which would tell you everything you needed to know.
Anyway, it's a great anecdote, even if it is apocryphal, and I've used it in conversation before because it is a really accessible way to get people to think about the problems of AI training.
Your approach will compute pixel importance in a single step whereas ablation based approaches generally need many passes.
