If every problem is bounded by a bunch of data points on a multi-dimensional plane, and some sort of clustering is done on them that reflects the problem domain, then isn't running through the samples during training equivalent to memorizing them and associating them and allowing for a small margin of error making it look like the model is able to generalize.
In short, doesn't that just boil down to how good we made our samples and setting the whole process in a way to memorize those samples ?
TB of original materials condensed into a few GB of neural network weight. If this is memorization, I would like it to be implemented into archive.org.
